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ABSTRACT 


Guidelines and recommendations are presented for the collec- 
tion of software development data. This guide discusses 
motivation and planning for, and implementation and manage- 
ment of, a data collection effort. Topics covered include 
types, sources, and availability of data; methods and costs 
of data collection; types of analyses supported; and warn- 
ings and suggestions based on Software Engineering Labora- 
tory (SEL) experiences. This document is intended as a 
practical guide for software managers and engineers, ab- 
stracted and generalized from 5 ynars of SEL data collection. 
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SECTION 1 - INTRODUCTION 


"Software engineering" is a term commonly accepted to de* 
scnue the way in which software dev^elopment oaqht to 
occur**ard the efforts to define and implement such a proc* 
ess. The 1968 NATO conference (at Garmisch, West Germany) 
popularized the term to serve as exhortation as well as de- 
scription (Reference 1). increases in software costs, in 
requirements for reliability, and in complexicy of system 
solutions involving computers all intensify the need for 
control and regularization of the software development proc- 
ess. What was arcane art in the early days of electronic 
computing must become a disciplined and predictable science 
if it is to meet the demands made of the computer industry. 

Transforming the software development process from art to 
engineering requires a disciplined evaluation of methods and 
practices, which in turn implies that some aspects of the 
process are measured. The intent of this document is to 
descr ibe--and to assist in — the process of extracting rele- 
vant and necessary data from the software development activ- 
ity. All phases of data collection are discussed, starting 
with reasons for collecting data, through types and sources 
and costs of data collection, to applications of the data 
once assembled. Although the specific experiences of the 
Software Engineering Laboratory (SEL) at the Goddard Space 
Flight Center (GSFC) (Reference 2) form much of the basis 
for the guidelines and recommendations herein, this guide is 
intended not as a historical review, but as a prescription 
for the future. 

This guide is aimed at software developers and managers who 
would like to be able to assess the value of present and 
proposed methodologies in their particular environments. 
Properly collected and analyzed, software development data 
can aid in identifying sources of problems and errors, in 



comparing costs of diffsrsnt sizes and types of projects, in 
making accurate estimates of production rates and project 
schedules. This guide is intended to support such efforts 
by providing a firm foundation for the collection of soft- 
ware development data. By suggesting approaches, by provid- 
ing a classification of types of data, by identifying costs 
and pitfalls, this guide should smooth the development of 
software development data bases and increase the value of 
software engineering efforts and analyses. The guidelines 
and recommendations in this document are primarily based on 
5 years of direct SEL experience in data collection, and are 
distilled from discussions at SEL workshops and elsewhere 
within the software engineering community. 

TO provide a common terminology and a ready reference, this 
guide includes a description and classification of software 
development data (Section 2) and an identification of 
sources of data (Section 3) . Guidelines for estimating the 
costs of data collection for different types and sources of 
data, including both direct and indirect costa, as well as 
suggestions for establishing priorities in a limited-budget 
environment, are provided in Section 4. The data collection 
procedure itself--collection by various means, processing 
the data, management of the data base, validation--is de- 
scribed in Section 5. A brief description of applications 
of the data is provided in Section 6 to demonstrate and sug- 
gest the uses of data in a software engineering setting. 
Specific recommendations for data collection methods, prior- 
ities, and applications are presented in Section 7. 

As noted above, the thrust of this document is more "how you 
can do it" than "how it is done here." Some review of SEL 
experience and procedures is in order, however, and is pro- 
vided in the appendixes. 
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1.1 MOTIVATIONS FOR COLLECTING DATA 


Althou 9 h the software community agrees that problems and 
shortcomings exist in the way softwate products are gener- 
ated, there is often great difficulty in specifically iden- 
tifying the problems and the means by which one should 
attempt to improve the process the next time. Each person 
has his/her own set of criteria for judging a software prod- 
uct (or process) to be successful or unsuccessful. Whether 
one looks at cost, productivity, reliability, modularity, 
document size, or other factors, many different parameters 
are used as evidence of whether a software project has bee^n 
successful or not. The level and extent to which these 
measures of software quality are used, or are meaningful, 
are certainly a long way from being commonly accepted by 
different software development groups. 

Before developers can attempt to improve the software 
development process and thereby the product, a clear under- 
standing of the strengths and weaknesses of the current mode 
of operation must be attained. Although it is somewhat 
unlikely, perhaps the current approach to developing soft- 
ware h'is no shortcomings and cannot be improved upon, if 
so, developers should be aware of this fact and should con- 
tinually reinforce the successful practices. On the other 
hand, if flaws and problems exist in the approach to soft- 
ware development, they must be identified before they can be 
corrected. That step is certainly the first in improving 
the software development process. This is the primary 
reason that anyone should want to collect software develop- 
ment data. It is the medium by which one can understand 
strengths and weaknesses, for only then can one aspire to 
improve the process and the product. 
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The motivation for collecting software? development data can 
be divided into two categorieit 

1. underatandinq , unlesa developers are completely 
satisfied with each software product gensiated, there is a 
need to understand the strengths and weaknesses of the proc- 
ess and the product. The arch. /ed information is the only 
means by which repeating poor practices can be avoided and 
effective development techniques can be reapplied. As soft- 
ware developers, the questions should always be raised, "How 
am I doing?" and "How can I do better?" 

2. Evaluation . The practices applied to developing 
software seem to be quite dynamic in that new methodologies, 
models, and tools are ccricincally becoming available, it is 
difficult to accept any approach blindly and to believe that 
it will have the stn>a favorable effects on the software that 
others may claim, as developers adopt newer and changing 
approaches, it greatly behooves the evolving process to con- 
currently measure the effects of particul.sr software devel- 
opment approaches. For example, with the availability of 
numerous and varied software resource estimation modelSr it 
is very problematical to select an approach that is applic- 
able and useful to one's own environment. Before adopting 
any one of the models, one would most certainly want to 
evaluate that approach utilizing development data within 
one's own software area. To evaluate the model, one must 
have access to the softwaie development data. This cer- 
tainly applies to the evaluation of any other facet of soft- 
ware engineering. 

Although understanding and evaluation are the two key moti- 
vators for collecting development data, there are other re- 
lated reaons. To experiment with varying development tech- 
nologies, data is needed; to justify changes to development 
plans (e.g., schedules or expenditures), data is needed ; to 
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provid* th« bails for hoping to advanct tha atata-of-tha-art , 
data is naada<V but thv lumnary fact that calls for tha data 
collaction affort is tha dasira and tha naad to undarstand 
and improva tha softwara dtvalopmant procass and product. 
Without accurata, dascript'.va data of tha softwara procass, 
thasa attampts at undarstanding and improving will ba futila. 

1.2 HOW TO USE THIS GUIDE 

Tha intant in producing this guida is to suggast answars to 
tha "How do X start?" and "What do I do?" quastions follow- 
ing tha racognition that tha softwara davalopmant procass 
can ba improvad. Tha answars arai "Undarstand what you'ra 
doing now" and "Collact ralavant data." This introduction 
will giva tha raadar a high-laval ovarviaw of tha procass 
without going into datails. Saction 1.4 particularly idan- 
tifias tha data to ba collactad and from whara it is 
collactad. Sactions 2 and 3 axpand on thosa topics for raf- 
aranca purposas and may ba skimmad or skippad on first raid- 
ing. 

Tha naxt quastion to ba askad oftan is "How much will it 
cost?"; tha raadar is diractad to Saction 4 for this an- 
swer. Noting the iterative nature of this entire process 
(saa Figure 1-1), Sections 4, and 6 may ba of value in 
any order. Saction 4 addresses "what (and how much)"; Sac- 
tion 5 treats the "how" of data collection; and Saction 6 
deals with "why." The details of managing and organizing 
the data are described in Section 5 (and treated elsewhere 
in depth) , along with tha basic guidelines for collecting 
data. 

As a reference to the process of data collection. Sections 2, 
3, and 5 form a unit. Section 2 describes the types of data 
in more detail than required in an overview; Section 3 de- 
scribes where the data is found; and Section 5 addresses how 
to get it. 
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Th« listed refsrsness ana the related pcalicatlons in Sec- 
tion i.5 suggest to the reader what can be done with the 
data once it is collected. 

1.3 ORGANIZING THE COLLECTION OF DATA 

Crucial to the success of the overall data collection proc- 
ess is a clear* advance understanding of the local purposes 
and analyses to be supported by the data ("local** meaning 
specific to tne installation sup^^orting the data collec- 
tion). The range and amount (and cost) of data to be col- 
lected must be directed and bounded by the uses to which the 
data will be put. Typical goals include 

e Quantifying the phasing and staffing patterns of 
software development projects 

e Numerically characterizing the developed software 

(e.g.* lines of code per module* percent comments* 
complexity measures) 

e Identifying major sources of errors (by phase* by 
activity* by personnel) and most commonly effective 
methods of detecting and correcting errors 

• Comparing methodologies* personnel* management 

techniques as factors in productivity differences 

These and other topics are treated briefly in Section 6 and 
discussed in detail in other publications (see References). 

Data collection must be identified a» one element in a co- 
hesive plan for software engineering analysis (Reference 3) . 
The goals of the entire effort must be clearly defined; 
these goals will be the driving factors in planning and im- 
plementing the data collection process. 

The first step* then* is the identification of these goals 
for the local environment. Specific* demonstrable goals 
should be set (e.g.* "Are there quantitative differences 
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between overall productivity measures of equivalent projects 
using FORTRAN or PL/1?" or "Do projects that have design 
walkthroughs have more or fewer problems during acceptance 
testing?") . 

Once the goals of the software engineering analysis effort 
have been determined, the specific data requirements of 
those analyses must be identifiefl. Sources, procedures, and 
costs all will affect the selection of data types, interde- 
pendencies must be identified to ensure that all requisite 
data are targetci for collection (e.g., resource data may be 
of little value witnout data on staffing patterns) . Sec- 
tion 4 covers these topics. 

For some types of software engineering analyses, quantita- 
tive models have been developed (Reference 4) . Where these 
apply to the specified goals identified, the types of data 
needed may be spelled out in the descriptions of the mod- 
els. For other analyses, identification of data items will 
involve more extensive planning. More detail is provided in 
Section 6. 

Cost factors, model requirements, data dependencies, and 
data availability may mandate review and revision of the 
software engineering analysis goals. An iterative process 
(as shown in Figure 1-1) may be required to resolve ques- 
tions of data availability, priorities, and budgetary 
limits. An evaluation of the anticipated return (e.g., more 
reliable products, more confidence in project estimation) 
should be performed as a justification for the data collec- 
tion process. Failure to recognize and secure the level of 
commitment required may jeopardize the software engineering 
engineering effort when the costs become apparent before 
those benefits can be demonstrated. 
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Figure 1-1. Data Collection--Functional Relationships 
and Iterative Processes* 


Definition of the data collection procedures to be used is 
implicit in the cost analysis. Data that is readily avail- 
able in machine-readable form (e.g., staff charge hours, 
computer use records) is inherently less expensive than data 
collected on forms or by interview. Th^ details of the col- 
lection procedure remain to be worked out, but the general 
sources and methods are defined in the early planning stages. 
Section 5 provides guidelines on the planning and design of 
the data collection process. Examples of forms used in data 
collection by the SEL are shown in the appendixes. 

Implementation of the system, especially in those aspects 
that directly impact the software development process (e.g., 
requiring programmers to fill out forms) , will require 
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planninq, public relations, and support from management. A 
phase-in period must be anticipated, both for che technical 
personnel and for the data collection procedures. 

1 . 4 OVERVIEW OF SOFTWAPP DEVELOPMENT DATA 

This subsection provides a high-level overview of the types 
of data pertinent to a software engineering effoil and of 
the relevant data collection methods. Sections 2 and 3 ex- 
pand on these topics; this overview is intended to provide 
an introduction and a common frame of reference and termi- 
nology for the sections that follow. 

In the context of data collection, software development can 
be viewed as consisting of five elements; the problem to be 
solved, the reso irces needed for solution, the environment 
in which those resources are applied, the process of apply- 
ing them, and the products genera,ted. Data must be col- 
lected characterizing each of these elements (at least to 
some elementary level) to support a responsible analysis 
effort. The data-collect ion process described in this doc- 
ument will be defined in terms of these five elements, de- 
scribed as follows: 

1. Problem data--the problem as described in the re- 
quirements specif ication, constraints (such as 
space or execution time), stability of the specifi- 
cation, type of changes, and so forth 

2 . Resource data--such as staff-hours and computer time 

3. Environment data--character iatics of the installa- 
tion which are relatively stable and invariant from 
project to project 

4. Process data--the methodologies, tools, techniques 
used in developing software 
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5. Product data--mea3ur es and metrics which character- 
ize the software and documentation (size, quality, 
complexity, etc.) 

Each of these classes of data is further categorized by re- 
porting interval as summary or snapshot (i.e., at completion 
or in process) data, and hy level of detail (from project or 
task level down to detailed component or module data) , Sec- 
tion 2 provides the formal definitions and classification 
scheme . 

Four major sources of software development data are de- 
scribed in Section 3. An understanding of the sources and 
terminology will be useful throughout this document. The 
four sources of data are as follows: 

1. Technical staff (i.e., programmers, designers, an- 
alysts, operations and maintenance people) -- 
information available via forms, interviews, activ- 
ity logs 

2. Managers--forms, interviews, personnel records 

3. Computerized records--accounting data (machine 
time, number of runs), configuration control rec- 
ords (e.g., PANVALET), transaction records (if 
maintained for detailed analysis) 

4. Products--source code, documents, design (may re- 
quire tools such as a source analysis program for 
data reduction to a usable form) 

1.5 RELATED PUBLICATIONS 

This subsection provides a list of particularly relevant 
publications that may aid in the collection of software de- 
velopment data. A more extensive list is provided in the 
list of references. 
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i. The Solitware Engineering Laboratory # V. Basili, 

M. Zeikowitz, F. McGarcy, R« Keitec, W. TruszKowski« 
U. Md TR77-5J5, and NASA-NSG- 5123 . Describes the 
background^ goals, and Intended data collection 
procedures ot the SEL as oC 1977. 

Tutorial on Mod. Is and Metrics for Software Manage * 
ment and Engineering , V. R. Basili, IEEE, 19B0. 
Describes purposes and methods ot software data 
collection . 

3 . Software Engineering Laboratory (SEL) Data Base 
Organization and User*s Guide , D. Wyckoff, 
CSC/SD-81/6011UD1 , Provides a detailed description 
of the data management and dissemination procedures 
of the SEL. 

4 . Software Engineering; Concepts and Techniques , 

P. Naur, B Randell, J. N. Buxton (eds.) Petrocelli/ 
Charter, New York, 1976. A conference report on 
tne 1968 NATO Conference on Software Engineering at 
Garmisch, West Germany. Contains many seminal 
papers of the field, stil,’. highly relevant. 

5 . Software Engineering Laboratory (SEL) Data Base 
Maintenance System (DB.^iM) User’s Guide and System 
Desct ipt ion , D. Card, CSC/SD-81/6079 . Describes 
the interactive data entry and editing system used 
by the SEL. 
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SECTION 2 - CLASSIFICATION OF SOFTWARE DEVELOPMENT DATA 


Software engineering data, as the term is used here, encom- 
passes any information that describes the process or prod- 
ucts of a software development effort and that is collected 
to support analyses and evaluations of that effort. Rele- 
vant data may pertain to any of a multitude of factors and 
be characterized by varying measures of objectivi*'y and 
level of detail. Productive research and analyses at a par- 
ticular facility will require a broad range of types of 
software development data to support comparisons and evalua- 
tionp This section describes several classes of data that 
together provide the range of data types required. 

The classification scheme presented here is intended to 
provide a common framework for discussion; other classifica- 
tions can and have been developed (Reference 3) . To be 
useful, such a scheme must organize the data in some logical 
relationship to the goals of the research effort. While 
clear and distinct divisions cannot always be drawn, a clas- 
sification scheme can aid in the research design by ensuring 
adequacy of data collection and clarifying data dependen- 
cies. The scheme presented here is intended to support 
research in improving the process and products of software 
development and to clarify the approach of the SEL to this 
research. The classes of data identified in this scheme are 

• Problem data 

• Resource data 

• Environment data 

• Process data 

• Product data 

Section 2.1 describes the dimensions of the classification 
scheme (subject matter, time of collection, level of detail) 
and relates this classification to the types and levels of 
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analyses that may be required. Section 2.2 defines the 
problem data class that describes the Initial requirement 
for software and the changes and constraints involved in the 
development. Section 2.3 defines the resource data class, 
which is used to track and summarize the expenditures of 
staff timt and computer time in the solution of the problem. 
The actual software development activity is characterized by 
the environment and process data classes (Sections 2.4 and 
2.5), which deal with such items as invariant attributes of 
personnel, management procedures, languages, hardware sup- 
port, methodologies, and standards. Section 2.6 defines the 
product data class, which describes the output of the soft- 
ware development activity. Section 2.7 describes change and 
error data--a composite of product and process data. 

2.1 CLASSIFICATION STRUCTURE 

For purposes of analysis, it is useful to be able to map 
classes of data onto elements of the software development 
model used in the analysis. In its simplest form, the model 
used by the SEL identifies an input (the problem or require- 
ment) , a process (software development methods and tecn- 
niques) , the environment (including types of machine and 
personnel resources available) , an output (the products-- 
software and documentation) , and some measure of the re- 
sources utilized. Figure 2-1 shows the basic elements of 
this model. 

To some extent, the mapping of data elements onto model ele- 
ments is determined by the purposes and perspective of the 
analysts. From a process measurement viewpoint, it may be 
useful to view resources and problems as input that is proc- 
essed to output a product. From a different standpoint 
(e.g., contractural) , "resources expended" may be considered 
a product to be controlled by manipulating the process. In 
the same vein, change and error data can be considered to be 
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Figure 2-1. Research Model 

either a product measure or a process measure^ depending on 
the analyses being performed. Because of this ambiguity, 
the data classification scheme described here has been made 
flexible to accommodate a variety of investigations. 


A rigorous analysis of some facet of the software develop- 
ment process will require some data on each element of the 
model--that is, some level of detail on the problem, re- 
sources, environment, process, and product. A study of pro- 
ductivity (products/resources) must consider as well the 
impact of variations in problem and environment and proc- 
ess. The ma^or dimension in this classification scheme is, 
accordingly, sub;;ect class of data. 

Two other dimensions {Figure 2-2) are used in classifying 
data; resolution and project status. Resolution, or level 
of detail, ranges from the broad brush-strokes of project 
overview oata (executive summaries, etc.) to the precision 
of module-level and component-level detail. Low-resolution 
data, such as total lines of code or total staff-hours for a 
pro]ect, IS useful for screening and comparing projects. 
High- resolution data, such as the purposes and results of 
indiviaual computer tuns and changes, provide insight into 
why a pro]ect progressed as it dia. 
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Project status, the third dimension in Figure 2-2, refers to 
the stage of the project or component about which the data 
has been collerted. "Predictive" data is available before a 
project or phase begins, or before a component is fully de- 
fined. "Snapshot" data reports status at some Intermediate 
point in the process. “Summary" data characterizes the 
project at completion of each major stage of development. 
Snapshot data thus describes the path from start to finish, 
or on this dimension of data, from prediction to summation. 

Table 2-1 briefly shows the classification scheme described 
above and the sources of data in each class and subclass. 

2.2 PROBLEM DATA 

The driving force behind the software development effort is 
the software requirement which, in its original statement 
and the changes and corrections that Inevitably occur, 
largely defines the scope of work in a development effort. 

The problem class of data is used to characterize the soft- 
ware requirement. The parameters used in this characteriza- 
tion are listed in Table 2-2. It should be noted that the 
major attributes of the problem are ite size and the stabil- 
ity of the requirement. The latter attribute is particu- 
larly time-dependent (i.e., snapshot data) because the impact 
of changes depends greatly on the timing of those changes. 

Problem data may also include constraints such as space or 
execution speed of the software or externally imposed dead- 
lines (such as defined by a spacecraft Launch date) . 

2.3 RESOURCE DATA 

Life-cycle studies and detailed project performance analysis 
require substantially more detailed information than is pro- 
vided by project- level summary data. Characterizations of 
actual effort expended on each phase or component of a sys- 
tem generally require that timely, detailed data be collected 
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Tabla 2 - 2 * ProbLam Data paramatara 


• problam Statamant 

Numbar of spaclf icationa 

- clarity of apaclf ication 

Natuca of apaci f ication (a.g., machina 
procaaaabla) 

• problam Stability 

• Number and timing of changaa to apacif icationa 

- impact of changaa (coat, perturbation of 
product) 

- Nature of changaa (e.g., correction, 

eYihancement , coamet ic, 

• Problem Character iatica 

Magnitude of problem 

- Perceived reliability of apaci f icationa 

- Complexity 

> Similarity to previous problems 

Constraints (calendar time, machine resources, 

interface to existing software) 

• Product Delivery Requirements 

Formality of documentation (especially 
transition documents) 

Reporting and review procedures 

- impact of software development data collection 
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from development personnel and/or from tne macnine account- 
in 9 systems. Required data are 

e Personnel resources applied 

- Management 

- Technical 

- Support (clerical^ publications, etc.) 

• Machine resources required 

Computer time used 

Terminal use (or other access records) 

- Data storage (e.g., disk utilization) 

Summary data may be used to perform comparisons and evalua- 
tions of projects; detailed resource data is necessary to 

t 

understand the differences and to fit individual tasks to 
models that can be used predictively. Personnel resources 
snould include information on the type of effort and compo- 
nent (if any) supported. Machine resources should be col- 
lected in accordance with whatever cost or chargeback algo- 
rithms are in use, with additional information to identify 
user, type of access, purpose (e.g., of a test run), and 
component (8) involved. 

2.4 ENVIRONMENT DATA 

With respect to software development, che environment con- 
sists of the relatively invariant faCi,ors of staff experi- 
ence and ability, computer system avaiiaoil i ty , management 
procedures, and similar attributes of a software development 
facility. Other factors may also be included, depending on 
the particular installation. In the Flight Dynamics area at 
GSFC, for example, the programming language--FORTRAN--and 
the graphic executive support system (GESS) are part of the 
environment. The distinction between environmental data 
(which characterizes longer term tendencies, factors, and 
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attributti) and procaaa data (wnich la concarnad with spa* 
cific projact-ralatad tools and tt^athodologias) is highly 
lnstallation*dapandant . 

The factors which ara traatad as anvironmant data includa 

• Computar languaga(s) usad 

a Staff compatanca 

• Sts'*! axparianca with typical problams 

a Staff axparianca with host, targat computars 

a Stability of softwara anvironmant 

a Availability of machina rasourcas 

a Stability of machina rasourcas 

a Staffing pattarns and taam organization 

a Managamant compatanca 

a Managamant axparianca with typical problams 

a Support facilities (a.g., librarians, technical 

publications expertise) 

2.5 PROCESS DATA 

The specific tools and methodologies that may be applied on 
a project-by-project basii are described by process data. 

For example, to test the "chief programmer/taam" methodol- 
ogy, a single project may be organized in this manner. Be- 
cause this procedure may be imposed on one project and 
removed on the next, the chief programmer/team technique is 
a process factor. On the other hand, a decision to switch 
to a new computer system, operating system, or language usu- 
ally cannot be a<s easily reversed; for better or worse, the 
change will probably impact a number of projects. Such a 
change would be treated as environment data. 
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Prom the standpoint of software engineering analyses, envi* 
ronmental data describes those factors which must be can- 
celed out of the results. (That is, the effects of the 
environment must be identified, accounted for, and disre- 
garded so that other effects can be analyzed.) Process data 
describes the tools, methodologies, and techniques which are 
being evaluated and undergoing experimentation. Table 2-3 
list^ some of these factors. 

2.6 PRODUCT DATA 

A tremendous amount of objective data can be derived from 
the products of the development process, particularly by 
analysis of the actual software. Elaborate and ambitious 
models of software development have been based on such anal- 
yses (e.g., Halstead's "software science,” Reference 5). 

The development products that can be so analyzed include the 
software- source code, design and specification documents, 
process documentation (e.g., design notebooks), and product 
documentation (user's guide, system description), as listed 
in Table 2-4. 

An important characteristic of product data is that it is 
relatively nonvolatile. Time-based information is required, 
of course, such as growth histories and implementation pat- 
terns, but significant amounts of data can be acquired as 
late as the end of a project. Change and error data, in 
contrast, are very clependent on timely recordation. 

2 • 7 CHANGE AND ERROR DATA 

Although change and error data does not form a distinct cat- 
egory (in terms of the proposed model of software develop- 
ment) , it is sufficiently important to software engineering 
(reliability) to be described separately here. From the 
standpoint of operations and maintenance, changes and errors 
are a product measure. During development, change and error 
data may also describe the process. 
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Table 2-3. Process Data Factors (Representative) 


Graphic representations and expressions 

• Flowcharts 

• HIPO (hierarchy- input-process-output) charts 

• Data flow diagrams 

• Hierarchy diagrams (baselines, tree charts) 

• System verification diagrams (SVDs) 

Development methodologies 


• Top-down design (stepwise refinement) 

• Top-down dev.fiopment (stubs, drivers) 

• Structured programming 

• Standards and protocols 

• use of librarian 

• Chief programmer/team approach 

e Unstructured ("egoless”) team 

Quality assurance mechanisms 


• Design, code reading 

• Design, code walkthroughs 

• Traceability analyzers 

• Standards compliance auditors 

• Verification, validation teams 


Configuration control and management 

• Source code control library system 

• Design, development notebooks 

• Change reporting and control protocol 

• Milestone charts 

• Configuration reporting tools 
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Table 2-4. Product-Derived Data 


source Code 

• Number o£ modules 

• Component coupling and connectivity 

• Component size 

Various measures of lines of code: old versus 

new, developed versus delivered, with and 
without comments, executable and specification 
statements 

Memory requirements: words of machine code 

• complexity of code (e.q., McCabe, Page measures) 

• Halstead's "software science" metrics 

• Execution characteristics 
Specifications and Requirements (if automated) 

• Traceability analysis 

• Change records (e.g., changes to specifications) 

• Complexity measures 
Documentation 


• Page counts 

Text 

Figures 

Copies of listings 

• Change history (from notebooks) 
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Users of an interactive deveioment system, with the instan- 
taneous turnaround and immediate response of a terminal- 
oriented environment, may use different testing strategies 
than would a one-run-per-day batch us»c*r. The change history 
IS different because the interactive user can afford to make 
one correction per pass, whereas the batch user must attempt 
to correct all errors for each new submission. Change and 
error data can be a characteristic of the process, or even 
of the environment. 

Prevention and correction of errors and adaptability to 
change are central to the efforts of software engineering. 
The analyses that can measure the effectiveness of various 
methodologies in these lespects require substantial amounts 
of data on the occurrence and processing of changes and (as 
a subset) on detection of errors. This type of data is of 
sufficient importance and volume to justify its independent 
classification, even thougn there may be overlap with re- 
source data. 

The definition of changes (and errors) should be related to 
the configuration management tools and procedures employed 
in the development effort. At each stage ot the development 
cycle, changes would be recorded for all items that had been 
accepted for control. The SEL uses the software module as 
the basis for defining changes, and relates code changes uo 
modifications in specification or requirements (as applic- 
able) . Change reporting and control is important throughout 
a software development task; tne emphasis and depth of cov- 
erage will depend on the specific goals of the research. 

Many of the methodologies and techniques being investigated 
in the field of software engineering are directed toward 
maintaining flexibility and integrity of a development 
effort in the face of changes (whether internal or exter- 
nal). Top-down design, iterative refinement, structured 
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programming, specification Ianguages*-all are partly moti* 
vated by this concern. To measure the impact and frequency 
of changes, data should be collected as changes are identi- 
fied and should include type of change, source, means and 
resources to implement, and magnitude and impact of the 
change . 

For the purposes of software engineering analysis, errors 
are perhaps the most interesting type of change because of 
the desire to limit error occurrence by applying appropriate 
techniques. Methods of detecting, preventing, and correct- 
ing errors are of concern to software engineering. A vari- 
ety of models of error occurrence have been devised to aid 
in predicting the number of errors, or errors remaining, in 
a software system. Data to support the use of these modijls 
and analyses is included within this class, information 
should be collected on a timely basis to identify the source 
of errors, the means used for error detection, and the type 
of error. As with other change data, records of resources 
required and impact of the error should be collected. 
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SECTION 3 - SOURCES OF SOFTWARE ENGINEERING DATA 


Sources, availability, and methods of collection of software 
development data are of major concern to the structuring of 
a data collection effort. Reliability, accuracy, consist- 
ency and completeness all have impact on the types of anal- 
yses that can be performed, and all are affected by the 
source of the data. Particularly important is the avail- 
ability of the same (or closely related) data from disparate 
sources, for purposes of cross-checking. This section de- 
scribes the primary sources of software development data, 
the types of data available from each source, and, briefly, 
the means of collection from each source. (More on the last 
topic is found in Section 4.2.) 

The primary sources of software development data are 

• Developers (personnel) 

Technical staff: analysts, designers, pro- 

grammers 

Managers 

• Computerized records 

Routine accounting records 

Special-purpose activity monitoring 

e Products: source code, documentation, specifica- 

tions 

3.1 TECHNICAL STAFF 

As used here, the term "technical staff" refers to all per- 
sonnel who contribute directly to the project products. 

This catchall term is used to distinguish productive effort 
from managerial activities and includes the designers, pro- 
grammers, quality assurance (QA) staff, test teams, and sup- 
port personnel. To the extent that managers contribute to 
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production, they also are included. (See also Section 3.2.) 
Because the technical staff is the major source of nonhard- 
ware resources that go into a project, they must also supply 
much of the detailed information on the process of software 
development . 

Although automated data collection procedures should be used 
when possible, more intrusive methods--forms , interviews, 
activity logs--will commonly be required. The costs and 
impacts of various procedures are discussed in Section 5. 

Table 3-1 lists the major data items that may be collected 
from the technical staff. The major classes of data are 
resource data (hours expended, computer usage) , product data 
(including subjective data, such as perceived complexity, 
which is not available elsewhere) , and change/error data. 

3.2 PROJECT MANAGEMENT 

Although the technical staff is the most valuable source of 
data concerning the detailed evolution of a project, more 
comprehensive and global project data should be obtained 
from project management. Administrative data (such as the 
amount of time actually charged to a project) can be used to 
cross-check (and fill gaps in) the more detailed resource 
data supplied by the technical staff. Being less absorbed 
in the details of a software development effort, managers 
are better able to provide evaluations of the project as a 
whole. Sensitive data such as experience and competence of 
the technical staff may not be available below the manage- 
rial level. Data on support facilities that may be shared 
among projects (e.g., typing support) is also available from 
project management. 

Table 3-2 details the types of software development data for 
which project management is a source. Note that, in many 
cases, these data can provide verification of data provided 
by the technical staff. 
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Table 3-1. Data From Tachnical Staff 


Staff Hours Expended (resource data) 


e By activity 

Design, code, test, travel, writing, etc. 
e By phase 

Requirements analysis, deaivjn, code and test, 
integration, acceptance 

e By component or subsystem 

Component Descriptions (product data) 

e Subjective measures and predictions 

- Type of software (algorithmic, I/O,....) 
Complexity, difficulty 

- Characteristics of specifications, design 

e Status, size at each phase 

e Relationship to other components 

e Constraints (memory, execution speed) 

# Language used 


Computer usage Records 


e Purpose and status of each run or session 

• Components involved 

e Characteristics of computer use (batch, interac- 

t ive . . . . ) 

Change Error Data 

• Origin of change 

• Components affected 

• Source of error, how (when) discovered 

• Time required to effect change 
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Table 3>2. Data From Managers 


Staffing Patterns and Characteristics 

• Number and phasing of personnel 

• Project organizational structure 

• Quality and level of experience 

Product Characteristics 

• Quality of products 

Reliability, maintainability 
Efficiency, effectiveness 

• Degree of structure 

• Readability (especially documentation) 

• Compliance with specifications, constraints 
Development (Methodology) Data 

• Development environment 

Type of computer support 
Congeniality 

• Manageability (visibility) 

e Adherence to standards or guidelines 

Adequacy of standards or guidelines 
Compliance and enforcement 

• Methodologies used (degree used) 
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3.3 COMPUTERIZED RECORDS 


Most computer installations provide a means of automatically 
collecting and recording details on the use of the facil- 
ities. These records typically include information on each 
instance of use (job submission, interactive session) iden- 
tifying the user, the time, and the resources used. Al- 
though typically collected and analyzed for chargeoack and 
performance evaluation purposes, these records can also pro- 
vide valuable data on software development activities, in 
addition to project-level resource (cost) data, these rec- 
ords can provide a cross-check on programmer-supplied com- 
puter use data. The specific data available will depend on 
the type of accounting in use at a facility. 

A different type of accounting information is provided by 
"librarian" systems (e.g., PANVALET). These systems can 
identify source code and library utilization by project and 
in some cases by component, properly used, these systems 
can provide data on project size, growth history, and errors 
and changes over time. 

Table 3-3 lists the types of information that are collected 
automatically (for other purposes) and can be used in soft- 
ware engineering analyses. Principal motivations in using 
these data are their low cost and their availability (al- 
though some effort will be required in the data reduction 
effort) . Also of importance are the consistency and reli- 
ability of the data so collected and the negligible impact 
on the software development process. 

In addition to accounting and library system records, trans- 
action records may be collected specifically to aid in soft- 
ware development analysis. Such data collection requires an 
initial investment (for the development of software to col- 
lect such data) and continuing overhead costs, but may be 
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Table 3-3. Computerized Recordkeeping 

Access and use Records 

• Computer loading by time and phase 

- CPU, memory 

I/O, mass storage utilization 

e use characteristics 

Frequency of use by person over time 

- Type of use (compilations, editing, executions) 

e Printout volume 

Librarian Accounting 

e Size and number of modules 
e Change history 

e Growth history 

Automatic Collection of Data 

• computer use data 

Session purpose and status 
Patterns of edits, compiles, tests 
Modules involved in each access 

e Product data 

Change and growth history 

Design evolution (PDL, prologs, baselines) 

e Test history, status, results 
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capable of reducing the need for form- f ill ing or replacing 
some data collection forms entirely, ^t present, the SEL 
has only begun to investigate costs and potential. Table 3-3 
includes some of the types of data which are thought partic- 
ularly amenable to this type of data collection. 

The high cost of using data collection forms and the limits 
to data collection using these forms makes it desirable to 
pursue such alternatives, but no empirical recommendations 
can be made at this time. 

3.4 DEVELOPMENT PRODUCTS 

Many of the analyses that can be performed on software de- 
velopment data require detailed information on the products 
of the development process, including the documents and the 
deliverable software. This information should be derived 
from the products themselves to ensure accuracy and reli- 
ability. For the deliverable software, a software tool is 
extremely valuable in extracting such data. Manual methods 
may be required for special cases (e.g., segments of assem- 
bler code in a FORTRAN system) or non-machine-readable prod- 
ucts (such as a design document) . Table 3-4 identifies data 
elements derivable from the products of the development ef- 
fort. 

The most commonly used program-size metric is "lines of 
code," but there are several interpretations of this term. 
Comparability and comprehension both can be enhanced by 
using a source code analyzer program to compute line and 
statement statistics on project and component levels. These 
statistics typically include number of source lines, lines 
with comments, executable statements, and detailed statement 
type statistics. 

Specific software models (e.g., McCabe's complexity measure, 
Halstead's metrics) (References 6, 7) will require additional 
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Tabic 3 >4. Product-Derived Data 


Source Code Analysis 

e Size (various measures) 

e Statement type distribution 

e Module classification 

e Complexity analysis 

e Specific models 

McCabe 

Halstead 


Document Analysis 

e page counts by type of page, by volume 
e count of changes to specifications or requirements 
e Type of specification, design 


Compliance With Constraints 

e Execution time 

e Memory loading 

Loader maps 
Dynamic measurements 
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detailed analysla. Bacausa tha actual product la uaad, tha 
caliability of tha data is maximizad. 

Dynamic analysas ara mora difficult to obtain, although soma 
computar systams ara abla to collact data on mamory, davica, 
and CPU utilization to soma laval of datail. To maasura 
complianca with constraints such as axacution tima or mamory 
allocation, spaciflc data collaction proceduras may ba ra- 
quirad. Systam load maps or axacution analyzars may ba 
usabla or at least provide soma guidance, in moat cases, 
subjective managomant- laval evaluations of performance and 
complianca will ba adequate for tha purpos<)s of software 
development data analysis. 

Document analysis is less easily automated, although the use 
of word processing may facilitate data collection and ctatic 
analysib. Some normalization process may be required to 
provide comparability among different styles of document: 
pages of sample printout require much less preparation than 
text; complex figures require more. Collection of descrip- 
tive data i.: required to support staffing level analysis or 
prediction of costs for future products. 
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SECTION 4 - COSTS AND PRIORITIES 


Identification and prediction of coats are essential to the 
planning of any project. Because collection of software 
development data is inherently a long-term activity, cost 
identification is especially important to ensure that ade- 
quate resources will be budgeted. This section identifies 
the types of costa incurred in collecting software develop- 
ment data and suggests the magnitudes of those costs based 
on SEL experiences. Because the planning process will un- 
doubtedly involve tradeoffs, cost comparisons (Section 4.2) 
and data dependencies (Section 4.3) are also discussed. The 
tradeoffs made at a particular facility will depend partly 
on these cost factors and partly on the goals of the partic- 
ular software data analysis effort. Section 4.4 discusses 
priorities and recommendations for data collection in terms 
of those goals. 

This section provides practical guidance for planning a data 
collection effort by suggesting how to maximize the return 
for a given level of investment. A successful software en- 
gineering research effort requires only--and all of--thase 
data needed to support the desired analyses. This section 
supports that effort. 

4 . 1 DATA COLLI' TION COSTS 

There are four primary sources of costs to collecting soft- 
ware development data: 

• Impact on monitored tasks 

• Processing (verifying, storing, disseminating) data 

• Development and maintenance of support software 

• Analysis of data 

Table 4-1 illustrates these factors and the magnitude of the 
associated cost. Costs are normalized by relating them to 
the magnitude of the projects being monitored. 
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Table 4-1. Measuring Software Technology Costs (As a 
Percentage of the Tasks Being Measured) 


TASK 

SEL 

EXPERIENCES 

GOAL 

OVERHEAD TO TASKS (OEVELOPMENf PROJECTS! 
FORMS 
MEETINGS 
TRAINING 
INTERVIEWS 
COST OF USING TOOLS 

5 TO 16% 

6% 

DATA PROCESSING 

COUECTING/VALIOATING FORMS 
ORGANIZING DATA 
ENCODING INFORMATION 
DATA MANAGEMENT AND REPORTING 

10-12% 

6 TO 8% 

SUPPORT SOFTWARE 

6 MAN-YEARS, THEN 

Vi man-year per year 

DATA BASE SOFTWARE 
CODE ANALYZERS 
REPORT GENERATORS 
STAFF TRAINING 

1 MAN-YEAR PER YEAR 


ANALYSIS OF INFORMATION 

MEASURING METHODOLOGIES 
DESIGNING EXPERIMENTS 
DESIGNING ANALYSIS TOOLS 
DEFINING MEASURES 

15 TO 25% 

10% 










4.1.1 TASK OVERHEAD 


The data collection process adversely affects the monitored 
dev«>lopment projects (tasks) in several ways, depending on 
the types of data and the methods employed. Data collection 
forms have the greatest impact, particularly when used to 
collect component-level data. Forms used by the SEL (see 
Appendix B) are filled out by the technical staff on a 
weekly basis (to monitor resources) and for each change and 
each computer run. Additional time is spent on project- 
level quality assurance for forms data and in filling out 
forms for resource, component, and project summary data. A 
startup cost is also incurred in training development per- 
sonnel to use the forms (to ensure consistency) . 

Task overhead also includes time spent in meetings to define 
subjective project-level data and in interviews to collect 
background data on changes, errors, and procedures. 

Additional costs that may be less easily identified are due 
to the use of data collection tools (such as the PANVALET 
librarian system) , which may be used more extensively be- 
cause a project is being monitored. No attempt has been 
made by the SEL to isolate these costs. Mora sophisticated 
automatic data collection procedures have not been imple- 
mented by the SEL. 

On tasks monitored by the SEL, overhead costs chargeable to 
the development tasks ranged from 5 to 15 percent of the 
total task cost. It should be noted that the SEL is col- 
lecting a very broad spectrum of data; nevertheless, it 
seems unlikely that the cost could be held under 5 percent, 
even with a less ambitious data collection effort. 
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4.1.2 DATA PROCESSING COST 


A continuing staff effort will be required to perform the 
collection, quality assurance, validation, data entry, and 
reporting functions. Here again, data collection forms con- 
tribute moat significantly to the cost, while other types of 
data processing (such as source code analysis) contribute to 
a lesser degree. Data m. nagement costs (error correction, 
status monitoring and repotting) must also be considered. 

The experience of the SEL has been that the cost of process- 
ing and managing the data amounts to 10 to 12 percent of 
development task costs. By simplifying the forms and 
streamlining the process, this cost can be cut in half. 

4.1.3 SUPPORT SOFTWARE 

An initial investment in support software is required in 
those cases where data management involves a computerized 
data base. In any case, training and documentation coats 
will be incurred during the startup phase. Because of the 
iterative nature of the data collection and analysis process 
(Figure 1-1) , changes are likely to be required as the ef- 
fort progresses. 

The SEL chose to build a data management system tailored to 
our requirements as they evolved. Six staff-years of effort 
were required to develop the system to its present state 
(References 2, 11, 12). Maintenance of the software-- 
primarily enhancements to support new data types--requires 
1 staff-year per year. As the data management system and 
the collection process become more stable, this is expected 
to approach 6 staff-months per year. 

4.1.4 ANALYSIS COSTS 

The cost of analysis will depend on the availability of 
easily adaptable software for statistical processing and 
reporting. This cost element is extremely sensitive to the 
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types of analysis required and to the level of detail of the 
research. The SEL.. which is investigating a very broad 
range of concepts and factors, has experienced analysis 
coats that amount to 15 to 25 percent of development task 
costs. This figure probably represents an upper bound to 
this cost factor. The SEL expects to limit this to 10 per- 
cent in the future by eliminating unprofitable studies and 
avoiding dead ends. An effort of more limited scope would 
require a commensurately smaller investment. 

4.2 COST COMPARISONS 

For each class of data or data source, both startup and con- 
tinuing costs must be considered in making comparisons. The 
initial investment involved in building a source code ana- 
lyzer may be large, but the cost of use is quite small. 
Conversely, the cost of processing data forms remains high 
as long as data are being collected. Figure 4-1 diagrams 
the relative startup and continuing costs of data collection. 

4.3 DATA DEPENDENCIES 

The data collection process is intended to support analytic 
efforts CO improve the software development process. It is 
important, therefore, to collect all the types of data 
needed for a particular analysis. Some of the required 
pieces of data, however, may not be directly obtainable; the 
problem is more complex when the needed items must them- 
selves be derived from other data types before they are 
usable. This section focuses on some of the interdependen- 
cies of such derivative data. The SEL is currently investi- 
gating (through factor and cluster analyses) consistent 
formulations for such hard-to-quantify factors as quality 
and and maintainability. For the present, some simpler de- 
pendencies will be exposed. 

Productivity requires not only the obvious lines of code and 
staff-hours but also the life-cycle phases included in the 
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calculation. Comparison of coding-only productivity with 
full-development productivity does not reveal anything. 

Similarly, change data can be compared only if the data are 
normalized to a specific life cycle period. This is partic- 
ularly important when projects with different methodologies 
are compared. 

The most glaring data dependency concerns the experience 
level of the technical personnel. Current data indicates 
that this factor can swamp other considerations, some means 
(typically, data supplied by project managers) must be found 
to normalize project data for this factor before meaningful 
comparisons between projects can be made. 

4 . 4 PRIORITIES OF THE DATA COLLECTION 

Obviously the type of data and the detail of data to be col- 
lected depend on the objectives of the efforts as well as 
the extent of resources available to support the collection 
process. If one is merely interested in studying or devel- 
oping resource estimation models, the heaviest and possibly 
the complete emphasis would be placed on collecting detailed 
data representing the resource expenditures for a project on 
a daily or a weekly basis. In this case, one naturally 
would not be concerned with collecting detailed change and 
error data. 

In this section, we will attempt to generalize the relative 
importance or significance of information that could be ex- 
tracted from a software development project. These general- 
ities are based on the extensive experiences of the SEL 
during the past 5 or 6 years. It is assumed that the person 
interested in collecting the data does not have a singular 
area of study in mind (such as software errors only) , but 
that the reader has more general objectives as stated in 
Section 1.1: first, to gain a clearer understanding of the 

software development process in the particular environment 
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and second, to support efforts when attempting to make a 
rational judgment as to the methodologies and approaches to 
be used in developing software in future projects. 

In outlining this priority schedule, we refer to the classi- 
fication scheme described in Section 2. Here, there are 
five classes of data that may be collected (process, prod- 
uct, project, resources, and error data); and within each 
type of data, there is a varying level of detail that one 
may request. These levels of information are summarized in 
Table 4-2. 

Obviously, some of the information defined is nearly useless 
without some of the other information; for instance, it does 
little good to know how much we spend on building a product 
if we know nothing about the size or characteristics of the 
product under consideration. 

Based on the experiences of the SEL, the following priority 
scheme is suggested to anyone pursuing the general task of 
collecting software data for studying software development 
strategies, models, or tools. The derived priorities are 
based on relative usefulness and difficulty in collecting. 
The list is ordered from the most important or highest pri- 
ority (1) to the lowest priority (10): 

1. Level 1 of Resource Data . This covers the man- 
hours expended on the project, as well as the computer usage 
and general support hours such as technical publications 
hours, ODC technical hours, and librarian support time. 

This information is critical in evaluating the total cost of 
a project and the general profile or model of how resources 
were consumed. The data has been widely used in helping to 
evaluate and build models for future cost and resource esti- 
mation . 

2. Level 1 of Product Data . To support even the most 
elementary analysis of any software project, it is mandatory 
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Table 4-2. 


Class of 
Data 


Process 


Project 


Resources 


Change 


Product 


Levels of Detail in Gathering Various Types 
of Data 


Level-1 Level-2 

Detail Detail 


General description of 
requirement 

Standards used 

Tools applied 
Team organization 

Phase dates 

Development machine 
Development language 

Level of staffing 

Weekly manpower expendi- 
ture on project 

Computer usage by week 

Error information as 
discovered and fixed 


Detailed characteristics 
of methodologies applied 

Description of each 
phase of the life cycle 


Subjective quality meas- 
ures of project perform- 
ance 

Staffing details 

Environmental pertucba- 
t ions 


Manpower by component by 
phase 

Computer usage by run 
Change information 

Change history 


Project size 

Number of lines of 
code 


Number of modules 
Number of new lines 


Document size 


Individual component 
character istics 

Growth history 

Subjective rating of 
project 
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that the characteristics of the product be recorded. This 
basic information includes such things as lines of code« 
number of modules, size of documents, and amount of new code 
versus amount of reused code. This data is not overly dif- 
ficult to capture and may be extracted once at the end of 
the project, but it is required by nearly all meaningful 
analyses that could be performed on a software project. 

3. Level I of Chanqe/Error Data . The two greatest 
concerns of building a software product relate to cost and 
reliability. Nearly all measures of quality are based on 
these two factors. The basic error data help to character- 
ize the er ror-proneness and the reliability of the prod- 
uct; this information complements the Information collected 
that pertains to resource expenditures to provide the basis 
for characterizing the cost and the reliability. The most 
important information included here consists of error type, 
date that the error is found, cause of the error, and level 
of effort required to correct the error. 

4. Level I of Project Data . To compare characteris- 
tics or profiles of different projects, it is quite impor- 
tant to record the general project characteristics including 
phase dates, staffing characteristics, type of software 
being developed, and the manager's general view of the de- 
velopment effort. 

b. Level I of Process Data . If one hopes to measure 
or evaluate the effectiveness of particular approaches to 
developing a software product, one must be aware of the de- 
velopment characteristics or approaches (methodologies and 
techniques) used during the development process. This first 
level of process data must describe such characteristics as 
team structures, standards followed, testing strategies, 
documentation requirements, approaches to configuration 
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control, and mathodologies utilized. This information can 
be captured quite easily on this general high level. 

6. Level 2 of Resource Data . This is the first of the 
more detailed level of data that should be collected from 
the software project. This level includes the weekly de- 
tailed manpower expended on each component (subroutine, 
function, and so on) as well as the type of effort put forth 
on each component (such as designing, coding, or testing). 
This level of detail allows one to determine such things as 
the amount of design effort put forth versus the amount of 
code effort. It also determines the relative amount of time 
required for the development of each of the system compo- 
nents . 

The Level-2 resource data should also include the detailed 
usage (by run) of the support computer facility. Here, one 
captures the reasons that runs were made and the general 
profile of run results (such as number of successes or fail- 
ures) . 

7. Level 2 of Project Data . This information consists 
of the subjective information describing project character- 
istics as viewed by knowledgeable managers. Here, one de- 
scribes such characteristics as the quality of the product. 

8. Level 2 of Process Data . This information details 
the models, tools, and methodologies used in developing the 
product. Each phase of the software life cycle must be 
characterized with some selected rating for each of the ap- 
plied methodologies or tools. As opposed to the general 
description (for Level 1) of the environment and basic de- 
velopment philosophy, here each of the detailed methodolo- 
gies (as listed in Section 2.2) must be itemized. This 
information is quite difficult to normalize and is quite 
vulnerable to bias and outright error. 
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9. Ltvl 2 of Product Data . One* a ganacal dascrlp- 
tion of tha softwara product ia attainad (siza, amount of 
documantation, and to on ) , ona should than attampt to char- 
actariza on tha componant laval (such as siza and complaxity 
of aach componant). This information is ganarally obtainad 
at tha and of a projact, but tha application of this infor- 
mation has baan found to ba quits acadamic to data. 

In addition, ona could attampt to captura, for aach compo- 
nent, tha astimatad charactar istics of tha componant bafora 
it is davalopad and again aftar devalopmant is complatad. 
This information should provida insight into which typas of 
components we can astimate and which typas of componants ara 
most difficult to charactarize until thay ara complataly da- 
veloped. In the SEL, this particular information has been 
found to be relatively expensive to collect and relatively 
difficult to utilize effectively. 

10. Level 2 of Chanqe/Error Data . Once all of the 
error characteristics have been provided, the detailed level 
of data for the change/error information would include de- 
tailed descriptions and histories of changes made to the 
software. This data must be captured each time a modifica- 
tion is made to design, specification, or code. This data 
has been found to be quite difficult and expensive to re- 
trieve, and the useful application of it to date seems 
somewhat limited. 
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SECTION 5 - DATA COLLECTION PROCEDURES 


The heart ot any aoftwara anginatring raaaarch program la 
data collactioni tha continuing procaaa of coilacting, val- 
idating, preparing, and furnishing tha data required for tha 
intended analyses. The major shortcoming of most published 
models, predictions, and hypotheses in the software engi- 
neering field is the lack of reliable data to provide vali- 
dation. Only with a rigorous, aggressive program of data 
collection can software engineering efforts provide a demon- 
strable payoff to a software development organization, 
without substantiation by way of data interpretation, all of 
the ptonouncements and pontif ications of software engineer- 
ing theorists are merely unsupported opinion. 

The cost and complexity of this collection process is 
largely responsible for the paucity of data. Clearly iden- 
tified goals, careful planning, and systematic implementa- 
tion are essential, along with adequate monitoring to ensure 
that data collection goals are met. This requires a high- 
level commitment of rupport for the activity, and implies 
the establishment of some central group or organization with 
long-term responsibility and resources sufficient for the 
task. 

This section describes the actual data collection procedures 
that are necessary to support practical #md useful software 
engineering research. Planning for data collection is dis- 
cussed in Section 5.1. Section 5.2 describes the design of 
the process. The mechanics of data collection, for each of 
the sources of data identified in Section 3, are described 
in Section 5.3. Section 5.4 deals with the management of 
the data from collection through availability for research. 
Costs and priorities of data collection (a major concern in 
real-world endeavors) are discussed in Section 4. 
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5.1 


PLANNING OVERVIEW 


The t^Ianning pi:ocese for date collection, aa noted in r'ig- 
ure 1-1, includes the followingi 

e Define the goals of the effort ("identify produc- 
v'-.ive methodologies"...) 

e Identify the analyses required to achieve those 
goals 

s Determine what data are needed to perform those 
analyses 

e Decermine where the data are, how to collect them, 
how much they cost 

s Specify pr iorities*>what to collect fust, what to 
defer, what to ignore 

This procedure, driven by availability of data and available 
resources, should produce a list of what data are to be col- 
lected and whence to collect it. 

The Initial steps in planning for data collection consist of 
defining the goals and requirements of the software engi- 
neering effort. Because the types of analyses desired may 
require sp<«cific classes and level of detail of data, some 
focusing of effort should be performed early in the research 
effort. Some published models, for example, have precise 
and detailed input data requirements. A high-level descrip- 
tion of the types of analyses supportable through these ef- 
forts is given in Section 6; an in-depth discussion of the 
potential of software engineering is beyond the scope of 
this docunent. Once the specific goals of the data collec- 
tion activity have been defined, the mechanics of data col- 
lection and managen. t can be planned in detail. 

Planning the actual data collection is straightforward once 
the data requirements and availability have been defined. 
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Although a number of different functions can be defined# the 
actual implementation is not significantly different than 
managing any data base function. Details on data management 
are provided in Reference 13. 

The major activities in data collection are shown in Fig- 
ure 5-1. The planning for these activities must identify 
and define responsibility for: 

• Implementation 

• Data collection and support 

• Data management 

• Project management 

These responsibilities are detailed in Table 5-1. 

Implementation functions include design of procedures, forms, 
data flows, and protocols and implementation of the data 
base; these are essentially one-time startup activities. 

Data collection and support functions include supervision 
and monitoring of the data collection process, ongoing qual- 
ity assurance at the point of collection, and data entry and 
validation. Data management functions include definition 
and maintenance of the data storage, access, and retrieval 
procedures. The functions of project management of the ac- 
tivity can only be defined with respect to the organization 
involved, but will certainly include monitoring to ensure 
adequate, reliable data. Consistent and valid inf ormation-- 
especially when collected by forms--cannot be obtained with- 
out active management support to ensure compliance and 
cooperation from development personnel. 

The major non-staff resource required is some medium for 
archiving and validating the collected data. For small- 
scale data collection activities, this may simply require a 
file cabinet, data entry/process/correction log, and dissem- 
ination procedure. For larger operations, a computerized 
data base is (considering the subject matter) the obvous 
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Fiqure 5-1. 


Major Functions in Data collection 





Table 5-1. Data Collection Functions (1 of 2) 
implementation 

• Oesign/specify data flow for each type of data and 
source# Qk, monitoring 

• Design and validate data forma 

• Specify data log format and protocol 

• Write procedures guides# instructions 

Data Management 

• Design files# access mechanisms 

• Build data entry and validation mechanisms 

• Design/implement maintenance procedures 

• Develop data access# reporting procedures 

• Provide training and documentation 

• Oversee data entry# validation 

• Record# evaluate# direct correction of problems and 
errors 

• provide status reports 

• Identify/request/allocate resources 

• provide training for data entry personnel 
Data Collection and Support 

• perform data coding# checking# entry 

• perform maintenance under data base administrator 
direction 

• Generate routine reports 

• Perform consistency checks on data at point of col- 
lectiofi 
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Table 5-1. Data Collection Functions (2 of 2) 

• Perform data cross-checking and validation Under 

direction of data base administrator or data col- 
lection supervisor 

Project Management 

• Establishes data collection procedures with devel- 

opment projects 

• Obtains, allocates resources 

9 Directs activity 
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alternative. These functions (whether or not computerized) 
are discussed in Secwton 5.3. 

5.1.1 IMPLEMENTATION FUNCTIONS 

The responsibilities for devising and establishing the col* 
lection and monitoring procedures must be defined in the 
planning stage. Each step of the data collection process 
must be identified, and monitoring and QA proceoures speci- 
fied. Data collection forms must be designed, and forms- 
logging protocols established. A manual or protocol of data 
flow, data descriptions, instruction and responsibilities 
must be written. Where machine records are to be collected 
(e.g., accounting data), some regular procedure should be 
described to minimize delays and loss of data and to fix 
responsibilities. Procedures for data review at the point 
of collection (e.g., the staff person filling out forms) 
must be developed and responsibility assigned. 

5.1.2 DATA COLLECTION AND SUPPORT FUNCTIONS 

The data collection process must be conducted and monitored 
on a continuing basis; responsibility for this support func- 
tion should be established during the planning activity. 
Machine-supplied records must be requested, acquired and 
processed. Forms must be collected and logged in a timely 
fashion. Point-of-collect ion quality assurance must be 
established as a regular activity. Attempts to recover 
missing data must be made and followed up. Particularly 
important when several projects are active at once, a single 
central collection and monitoring function will smooth and 
simplify the data collection process. 

QA functions should be the responsibility of someone associ- 
ated with each development project and trained in the 
requirements of the software engineering effort. in this 
way, consistency of data within a project and compatibility 
across projects can be ensured. 


5-7 



Data entry personnel should be trained to become familiar 
with the format and typical content of data entry forms. 

This will minimize errors in data entry and provide an addi- 
tional check on information content. 

5.1.3 DATA MANAGEMENT 

The data which have been collected, reviewed, logged, qual- 
ity assured and assembled must be managed to facilitate ac- 
cess for analysis purposes. This is essentially a data base 
problem rather than a software engineering problem, and is 
not treated in depth in this guide. Some notes, however, 
are appropriate here. 

• The organizing (indexing) principles should reflect 
the data analysis requirements 

• The data management system should facilitate iden- 
tification of missing or incomplete data 

• Access procedures should protect any sensitive 
(proprietary; personnel;...) data without blocking 
access to the data base 

• Because data requirements change with increased 
understanding, the data management system must be 
flexible and amenable to re-organization 

5.2 DESIGN OF THE DATA COLLECTION PROCESS 

AS noted above, the data collection process involves col- 
lecting, validating, storing, and making available data re- 
garding software development efforts. The overall design of 
this process involves numerous elements, as shown in Fig- 
ure 5-1. The logical and physical organization of data must 
be defined; validation procedures must be specified; data 
storage and retrieval mechanisms (whether or not computer- 
ized) must be identified. The "what" of data collection is 
driven by the analysis requirements; the "how" depends on 
the environment and the available resources. The procedural 
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design (organization, validation, storage, and retrieval) 
follows from the answers to the "what" and the "how." as 
illustrated in Figure 1-1, the entire planning and design 
activity is inherently iterative. 

5.2.1 DATA ORGANIZATION 

The organizing principles chosen for the data base should 
simplify the collection and/or analysis activities. The 
SEL, for example, organizes its data by project and by 
form-type. Most SEL analyses are related to comparisons of 
different projects (to identify differences and evaluate 
methodologies) and are made easier by the project-oriented 
organization. Data validation procedures and manipulations 
are typically tied to the original forms (for verification 
purposes) , so the forms-type division is a useful one. 

Other organizations might be chosen (e.g., by type of data) 
as long as the storage method used adequately supports re- 
trieval for purposes of editing and analysis. 

The SEL data base organization was influenced somewhat by 
the limits of the computer system used for data base mainte- 
nance. As noted in Section 5.2.3 below, different storage 
systems may impose different requirements or, conversely, 
provide different opportunities. 

5.2.2 DATA VALIDATION 

It is assumed here that some quality assurance and validation 
takes place at the data collection point (Section 5.1.2). 

To ensure that the data stored is usable, accurate informa- 
tion, an additional validation stage is required in the data 
processing activity. The SEL data base is computerized, and 
can therefore make use of automatic validation programs to 
ensure completeness and consistency. Noncomputerized sys- 
tems would use manual techniques to ensure that all data is 
carefully accepted, logged, summarized, and stored for re- 
trieval. 
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This requirement for validation is, in fact, a significant 
argument for computerization of the data. Because all items 
are checked as they are entered, sporadic errors (which 
might slip past a spot-checking system) are discovered. 

When found, systematic errors (e.g., consistently entering 
an incorrect code for a field) can often be corrected en 
masse on a computerized system. 

5.2.3 STORAGE AND RETRIEVAL 

Design of the storage and retrieval system greatly depends 
on the available resources, possible methods range from 
simple file folders to elaborate data base systems. The 
chosen system will greatly impact the types and difficulties 
of analyses to be performed, with the cost of data entry 
(highest for computerized systems) balanced against the cost 
of repeated access and summation (highest for linearly or- 
ganized manual systems) . The intent of this section is 
merely to emphasize the importance of this design decision. 
More detail on how to design a system is given in Refer- 
ence 12 . 

5.3 COLLECTING THE DATA 

As part of the design and planning activity, specific mech- 
anisms of data collection must be identified. Collection 
methods ^'ave been devised for each of the sources described 
in Section 3. Table 5-2 lists these methods and the data 
sources for which each is suited. Advantages and drawbacks 
of each method are discussed below. 

In collecting (as in planning) the data, it should be kept 
in mind that accurate, complete data--even if collected at 
only a high level--is more useful than a large amount of 
data of uneven coverage and consistency. The data 
collection process should be directed at the information 
that is likely to be available, rather than at collecting a 
little of everything. 
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Table 5-2 

Source 

Development staff 

Machine records 

Development products 
Managers 


Data collection Methods 


Collection Methods 


Data forms 
Interviews 

Automated collection 

Data reduction and cross-correlation 
of automatically collected informa- 
tion 

Specifically designed analysis pro- 
grams 

Data forms 
interviews 
Consensus -forming 
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5.3.1 FORMS 


The bane of every programmer ' a existence {at least, for pro- 
grammers on monitored projects) , data collection forms are 
perhaps the easiest-to-implement method of collecting data 
from development personnel. 

Properly designed and managed, data forms can provide a 
wealth of information concerning the development process at 
both summeiry and component levels of detail. Forms can also 
serve the purpose of providing archival storage of ephemeral 
data (such as the purpose of a test run), permitting data 
collection to be uncoupled from the data processing func- 
tion. The iterative nature of software engineering re- 
search, of coarse, implies that this uncoupling could allow 
inconsistencies to occur in the data base. When new forms 
are designed with new questions, problems of compatibility 
may arise. 

There may be (as yet) no good alternatives to forms for 
collecting some types of data. There are, however, some 
serious drawbacks to the use of forms for extensive data 
collection. In addition to the impact on the development 
process and schedule, and the potential morale and compli- 
ance problems resulting from the drudgery and boredom of 
filling out forms, the design of forms is almost an art. 

It is the experience of the SEL that development personnel 
will complete progress/status/exception/etc. forms only with 
reluctance and with continuing prompting and exhortation 
from management. When such encouragement has occurred, de- 
tailed reports containing a wealth of data have been col- 
lected. Despite some grumbling and some often justifiable 
complaints about unnecessary duplication of effort, forms 
were completed usefully and consistently. With Lukewarm or 
sporadic management support, forms were completed cursorily 
or not at all. 
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Table 5>3 lists some of the desirable characteristics of 
data ccilfction forms. Although specific data requirements 
may contravene some of these guidelines, they serve as a 
target for the art of forms design. 

The data collection forms included in Appendix B have been 
developed and used by the SEL at GSPC. They are included as 
suggestions and as examples produced by a second- iteration 
forms design process. These forms reflect the research 
goals of the SEL and are included not as a prescription but 
as a guideline. 

5.3.2 MACHINE RECORDS 

Automatically collected records such as charge accounting 
data and source library update data form a major and gener- 
ally reliable source of information about the software de- 
velopment process. A major attraction is the fact that, by 
definition, these data are collected independently of the 
software engineering effort. The cost of using them is usu- 
ally limited to the cost of data reduction and correlation. 
The specifics of the data available will vary from one in- 
stallation to another because of differences in systems,- 
accounting software, and chargeback philosophy, and in 
source code library control systems. In general, however, 
these data can support at least summary-level analyses, and 
in many cases (and with greater effort) , component-level 
detail . 

Data reduction and cross-correlation efforts typically in- 
volve correlating account numbers, users, and job or compo- 
nent names to specific projects or systems and compressing 
masses of data into a usable synopsis. This may be accom- 
plished by the accounting software itself or may require 
development of analysis programs. Data security may also be 
a concern, depending on the facility. The coverage and 
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Table 5-3. Desirable Forms Characteristics 

• Keep Brief (1 side of 1 page) 

e use checkoff or unambiguous short answer 

e Provide space for comments 

• Ensure that all requested data can be justified 

• use terms which are familiar to the specific envi- 
ronment 

e Provide professional looking, quality reproduced 

forms (rather than typed and xeroxed) 

e use the same forms for all projects 

e For Repeated-use forms (e.g., status reports): 

limit requests to weekly 

do not ask for the same information repeatedly 

e Integrate forms into the development reporting 
process 
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accuracy of the data will ganarally justify ths investment 
in data reduction software. 

It would be desirable to use computer«collected records to 
profile all the parameters of software development. Impact 
on the development proces'i would be minimized, coverage and 
accuracy would be improved, consistency and comparability 
would be maximized. But the front-end investment in tools 
and procedures that would make this feasible has not yet 
been made. Computerized accounting records and source code 
control systems <see Section 5.3.2) provide some such data, 
but the potential for future initiatives in this area is 
large. Not all data can be collected automatically, but the 
limits of what is possible are presently unknown. More re- 
search and investment is required. 

5.3.3 AUTOMATED DATA ANALYSIS 

Much of the output of a software development effort is read- 
ily amenable to analysis by appropriately designed software 
tools. The software product itself is clearly the most re- 
liable source of data on product characteristics, although 
some condensation and interpretation of data may be neces- 
sary for application, where automated design and/or docu- 
mentation tools are employed, these products also can be 
analyzed with minimal investment. 

Typically, the analysis programs to provide these data will 
need to be developed specifically for the software engineer- 
ing effort. Each type of product (source code, program de- 
sign langukge, documents) will require a tailored analysis 
program. The fact, however, that these sources are fixed, 
definitive, and machine readable makes this approach highly 
attractive. 
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*>.3.4 INTERVIEWS AND CONSENSUS 


Som« typ«s of information arc not aasily collected on data 
formf> because of the subjective or imprecise nature of the 
data. For example, motivation for a particular design deci- 
sion may be useful in understanding a project but not easily 
recorded on a checklist, constraints are frequently defined 
in a complex fashion suitable for the development staff but 
not for data recording. Judgments on the acceptability, 
clarity, or maintainability of software are essential to 
software engineering analysis but difficult to quantify in a 
consistent manner. 

For these types of data, as well as for monitoring of the 
data collection process in general, interviews or meetings 
are the most direct and useful procedure. Responses that 
are not easily collected on checklists can be amplified in 
discussions between developers and data analysis staff. 
Consistent scales of evaluation can be developed in meetings 
of project managers with the software engineering team. The 
cost of such procedures will limit the extent of use, but 
some such activities should be anticipated and planned. 

5.4 DATA MANAGEMENT 

Managing the data after it has been collected and verified 
Involves data entry, data editing, validation, and an inter- 
face to the analysts who will use the data. SEL experience 
shows that collection of accurate and complete d^. . requires 
effort and experience. The data management system should be 
designed to support this process by identifying gaps and 
inconsistencies in the data (e.g., by using sequential form 
numbers) . SEL experience also shows that data requirements 
change with improved understanding of the development proc- 
ess. The data management system must therefore permit 


reorganizations data as new items are added and existing 
data proves unusable or valueless. 

The design and implementation of the SEL data base is de- 
scribed in Reference 12. 


SECTION 6 - APPLICATIONS 


The software development data base will facilitate many ap- 
plications of interest to managers and researchers. The 
manager would like to have monitoring, estimating, and eval- 
uating tools to examine ongoing software development tasks. 
The substantial variation in the results of software engi- 
neering experiments by different researchers suggests that 
the effect of the local environment on the development proc- 
ess is a powerful one. This can best be understood by as- 
sembly of a historical record of development efforts that 
can be used to "tune" the general models to local conditions 
(Reference 14). Sections 6.1 through 6.3 discuss the 
classes of tools that can be devised to take advantage of 
this historicl data base. Some of the software engineering 
research questions which can be addressed with the data are 
outlined in Section 6.4. The necessary data for all desired 
analyses should be included in the initial data collection 
plan. 

6.1 MONITORING 

A manager is likely to find the collected data very useful 
in keeping track of the status of ongoing software develop- 
ment projects. The three types of reports described below 
would be erpecially valuable. 

Resource utilization (computer time, staff-hours) can be 
tracked and displayed in tables and/or graphs. Comparison 
of these values with budgeted values can help monitor devel- 
opment costs. 

The progress towards completion of the development effort 
can also be tracked. The number of modules designed, number 
of modules coded, number of modules tested, and lines of 
code developed to date can be determined and report peri- 
odically. 
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review of design and code measurements made during the de- 
velopment process can be used to detect potentiiil problems 
such as unmanageably complex code, incor; '<%te design, and 
low testability. Those can be the basis c' a concurrent 
quality assurance program. 

6.2 LIFE CYCLE MODELING 

The goal of life cycle modeling is to relate the costs of a 
software development effort to its producti;. A great many 
models with different emphases have been devised (Refer- 
ences 4, 15, 16). However, as mentioned earlier, all such 
models should be cal..brated in the user's environment with 
historical data. Thus, one of the important applications of 
the data base is in determining appropriate model constants. 

The utility of life cycle models is as an estimation tool. 
They provide a method of estimating the cost and product 
size of a development project. 

6.3 METHODOLOGY EVALUATION 

Another application of software development data is in the 
evaluation of methodologies and environmental factors. De- 
velopment performance information can be used to identify 
the effect of various development approaches (top-down de- 
sign, structured programming, etc.) in the user environ- 
ment. Nonmethodological environmental factors (travel time, 
group size, etc.) can also be considered in these analyses. 
Thus, the user can develop a procedure for evaluating and 
ameliorating his/her software development process. 

6.4 RESEARCH 

Very few questions about the nature of the software develop- 
ment process have been answered definitively. Extensive 
work is cun rntly being done in the areas of software met- 
rics (Reference 7) , classifications, reliability (Refer- 
ences 17, 18), and models. Analysis in all of these areas 
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has been limited by the lack of substantial reliable data. 
The reader may wish to address him- or herself to some of 
these topics. 
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SECTION 7 - REt-OMMENDATIONS 


Perhaps the moat important recommendation is not to expect 
instant results, usable data must be collected over the 
life of a project, and data on a number of projects must be 
assembled. This does not happen c'ernight; as a result, the 
data collection plans must be laid out to allow adequate 
time before analysis results are expected. 

Beyond that, some thoughts from the SEL are as follows: 

1. Subjective management information is very important 
to project analyses and comparisons. 

2. DO not worry about the Hawthorne effect; typical 
projects are too long and typical programmers are 
too professional for psychological effects to have 
a significant impact. 

3. Try to provide feedback from the data collection 
process to the technical staff. 

4. Explain the purpose of data collectiot; to the tech- 
nical staff; try to elicit active support. 

5. DO not spend too much time demanding more, or more 
precise, or more accurate data. 
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APPENDIX A - SEL DATA COLLECTION EXPERIENCES 


Since its formation in i$T7 , the SEL has monitored more than 
40 development projects representing over a million lines of 
code. The projects include a wide variety of applications, 
computers, size, and complexity, but they are concentrated 
in tne flight dynamics software area. The major source of 
data consists of FORTRAN software systems averaging 
b0,000 lines of code that perform "scientific" data process- 
ing of spacecraft telemetry data. 

The SEL devised a set of forms to collect the data initially 
expected to be useful. These forms were revised extensively 
after a period of use and feedback from the programmers who 
filled them out. A second revision is currently being con- 
sidered for the next hintus in monitored projects. 


Data nave also been collected from the end-product source 
libraries using a static source code analyzer called SAP. 
This program has been modified to count and report the sta- 
tistics used by various complexity models and measures and 
to aid in selecting a most consistent definition of "lines 
of code. " 

Software currently under development by the SEL will analyze 
data from accounting tapes maintained by the primary comput- 
ing facility for monitored projects, an IBM S/360 facility. 


The SEL has also developed an extensi 
this data using a PDP-11/70 computer 
software, written in FORTRAN for RSX- 
ard data entry and editing capability 
formed by data entry clerks under the 
personnel . 


ve system for storing 
system. The data base 
llM, provides a stand- 
. Data entry is per- 
direction of SEL 


More detail on 


the SEL IS found in Reference 


2 . 
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APPENDIX B - SAMPLE DATA COLLECTION FORMS 


The forms reproduced here are used by the SEL at the Goddard 
Space Flight .enter to collect data on development proj- 
ects. The terms used in these forms are defined in Sec- 
tion B.2. 

B.l. SAMPLE DATA COLLECTION FORMS AND INSTRUCTIONS 

This section contains sample data collection forms and 
instructions for their use. The instructions precede the 
torms. The following forms are included 

1. General Project Summary (GPS) 

2. Resource Summary Form (RSF) 

3. Component Summary Form (CSF) 

4. Component Status Report (CSR) 

5. Run Analysis Form (RAF) 

6. Change Report Form (CRF) and Attitude Maintenance 

Change Report (ATM) 
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INSTRUCTIONS FOR COMPLETING THE GENERAL 
PROJECT SUMMARY • FORM 580>l 


11111 form IS usckl to Ltaisil'y iht proieii and wilt tx usti'i in coivunction with the other 
reporiutt lormt to mnsure the estimated versus actual devetopment propesa. It should be 
(Hied out by the piuiect manager at the beginning of the protect, at each mator miieslone. 
and at the end. Numbers and dates uied at the initiation of the protect arc assumed to be 
estimated; mtermediate reports should change estimates to actuals iif known) and update 
estimates. The (Inal report should accurately describe the system development life cycle. 

A. PROJECT DESCRIPTION 

Owcilptian. Give an overview of the project. 

Inputs. Speadcationt and requirements i etc.) of project Give the format of these. 
Requheiiienta How requirements are established and changed. 

lYoducit DevelopMl. List all items developed for the project te.g.. operational lystem, 
testing system, simulator, etc.). 

^oducts DeUveresJ. Last all items required to be delivered (c.g.. source of the oper- 
ational system, object code of the operational system, design documents, etc.). 

B. RESOURCES 

Target Computer System. System for which software was developed 
Devetopment Computer System. System on which software was developed. 

Conatrainis. Last any sue or time constraints for the 'iiiiihed product. Do you a'ltlc^ 
pate any problems in meeting these constraints’ 

Useful Items From Simitar Projects: 

I List previous projects, which will contribute vanous ispeus to this project. 

2. For each protect, give the percent of the current project it makes up in each 
of the 3 listed aspects. 

3. For each of the 3 listed ispucis ispecification. design, code) checK what level 
of modifications are necessaiv. 

C. TIME 

Start Date. First dale of work, including design and modHlcaiion of the specincalions. 
End Data Delivery date. 

Eatimaied Lifetime. Estimate the operational life of the wsiem. 

Miision Date. Scheduled operation Uaic ot the system iwriie unknown if not known or 
undecided yet on anv of these dates). Date project must be operational. 

Confidence Level. Give the percent probability you think the end date is realistic. 

(e.g.. 100*?' means certain delivery on that date. O'* means no chance ot delivery, i 
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D 


COST 


Com Total amoimi oi tnornty th« protect cuMa, inclutiiny both comraci and in>liouie 
aiati. 

Maximum -Kvaibble. Maximum amount available, imlepcmlcni ol «hat cMiniaied coxt 

IV 

Coiuldence Level. Rale percent reliability in coil eiiimate. 

How OeiermiiMi) M initiation How ii it eMimated. at completion how it ii calculated. 

Penonnel. Give the number oi lull lime equivalent pencni required at inception ol the 
project, 1 J ol Ihe way into the project, 1 3 of the way mio the project, at Ihe com- 
pletion of the protect. 

Total Perion Months tjive ilie total number of monihi llul full time equivalent per- 
toMiiel I managers, deugnen. programmers, key punchers, editors, secretanes, etc. i are 
assigned lo the project Do not include all overhead items such as vacation and sick 
leave. 

Computer Time. Give the total number ol hours on all systems normalised to one 
injehme le g., the IBN* .>b0 '5 1 and name the machine 

E. SIZE 

Sire <i( Ihe Syiiem. Include Ihe total amount ol machine space needed tor all instruc- 
iioiix generaicil .ui the protect plus Ihe space for data, library routines le.g. FORTR.-V.S 
I O package I and other code already .ivaiiable. Break down sire into data space and 
nstructioi space. 

(.oiUideiice Level. Rale percent rehabiliiv in sue estimates. 

Total Number of Source Siaietncnis. Gisc the number of FORTR.XN. .XLC. or any 
.'ihcr ljngu.ige instructions generated specillcally for this project. 

Structure of System. Give overall structure of system. Is it a single load module, is it 
n 'verlas siruciure or is n a set of independent programs ’ For oserlay and separate 
programs, give the numher and average sue of each. 

Define X our CuiKepi of a Module Give Ihe criteria you are using to divide the soft- 
ware into modules. 

Ecstiinaied Number of Modules. Include only the number ol new modules lo he vvriilen. 

Range in Module Sue Give the number of insiruciions in ihe minimum, maximum and 
average module and Ihe langiuge in which they are wriileri as a reference. 

.Nuiniser ut Dtffcreni LO Formats Used Give the number of distinct external data sets 
that are required lor the system including card reader, printer, graphics dev.ee. and 
lemporarv Tiles 

F COMPLTER VCCESS 

V .ibrarian is a person who can he used to perform anv oi the clerical 'unclioiis associ- 
ated with i-rogramming. including those given on ihe chan. Check Ihe appropriate hoses 
lor lliiise persons wno luic access to the computer to perform the given lunctionv Give the 
petceniagc ol lime spent hy each in batch and interactive access to the computer. 
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C. TECHNIQUES EMPLOYED 

For "Itvcl." ipccily to wtui l«vel of detail in the ilniehcd proicci the technique is used. 
(e.|., subroutine, module, tefinenti of 1000 lines, top le«el. etc.) 

SpactfUsltom 

Functional - Components are desenbed as a tel ol functMnt. each component 
performing i certain actloa 

Procedural - Components are tpeallcd m some algorithmic manner le.g., using a 
PDL) 

English - Components are specified using an English Linguage prose iiatement of 
the problem. 

Formal - Some other formal system is used to specify the components. 

Design and Dcsrelopment 

Top Down - The implementation of the system one level at a tune, with the current 
level and expansion of the yet to be defined subroutines st the previous higher level. 

Bottom Up - The implementation of the system starting with the lowest level rou* 
tines and proceeding one level at s time to the higher level routines. 

iterative Enhancement - The implementation of successive implementations, each 
producing a usabie subset of the final product until the entire system is fully 
developed. 

Hardest First - Tlie implementation of the most difficult aspects of the system first. 
Other - Desenbe the strategy used if it is not a combination of any of the above. 
None Specified • No particular strategy has been .'pecified. 

Coding. The l1nal encoding of the implementation in an executable programming 
language. 

Structured Code With Simulated Constructs - The language does not support slruo 
tured control structures le.g.. FORTRAN1 but they are vimuiaied with the existing 
structures, piease state the structured control ttraciures you are using (e.g.. WHILE. 
CASE, IF). 

Structured Control Constructs • The language supports structured control struc- 
tures le.g., a FORTRAN preprocessor) please list structures you are using. 

Other Standard - Desenbe any other standard you are using. 

None Specified - No particular strategy has been specified. 

Validation/ VerinoBiion. Testing execution of the system, vu a set of test cases. 

Top Down - Stubs or dummy procedures are written to handle the v et to be imple- 
mented aspects of the system and testing begins with the top level routines and 
proceeds as new levels are added to the ivsiem. 

Bottom Up - Check out of a module at a time uime test drivers and starting at the 
bottom level modules fint. 
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S<rtuiiire Driven - Lting virticlure ol prug.-am to Jelerinine irvi Jjle te g . every 
vuiemeni ol program execuieU it lean once). 

Speoikation Dnven - living vpeciiicaiioni ol program lo determine levi data le.g . 
alt input output relaiiontliipt hold for a tel of teti data) 

Other - Oetenbe any other tiraiegy you are uaing 

Vine Specilleii - No letling ttraiegy hai been tpectlleii. 

ValiUaiion Verincsiion. Invpeciion vuual examination of the oode or devign. 

Code Reading • Vitual mtpection ol the code or deiign by other programmer!. 

Walk riiroughv - Formal meeling vettiont lor the review ol code and design by the 
kariouv members ol the project, lor technical rather than management purposes. 

Proott - Formal prooltol the design or cx>de. please ipecify the techniques used, 
e g.. axiomatic, predicate iransl'orms, lunclional. etc. 

None Specified - No <ntpec'ion techniques have been tpecined 

Tliere is vome vpace given to permit the luriher explanation ot any ol the strategies tliat 
nuy be used. 

H. FORMAL notations USED AT VARIOUS LEVELS AND PHASES 

C>ive the phases le.g.. design, implemeniaiioii. testing, etc. I and levels Isubrouline. 
module, segments ot 1000 lines, lop level, etc I at which any type of formabsm tHowchari. 
POL. del will be iised in the development ol the system. 

I vltomated tools used 

Name all automated tools used, including automated versions ol the formalisms given 
above and .ompilers lor the programming languages used, and at which phase and at what 
level thev arc used. Include any products that may be developed as part of this project 
ic g.. simulator I. 

j organization 

Describe how the personnel are subdivided with respect to responsibilities into teams 
or groups, giving titles, brief job descriptions, the number of people satisfying that title and 
tlieir names and organuational affiliations if known. 

K STANDARDS 

List alt standards used, whether they are required or optional, and the title of the 
document dcscnhing ilie standard 

L milestones 

itive tiie phase at wluch management may check on progress ot the development ol the 
yy >i.'m le.g.. -peciiicJtion. design, implementation of version I.eic.i Stale also the date at 
which It yiiould take place lat cc'mpletion ot the project*, how it is to be determined that the 
milestone was reached, who will be responsible tor reviewing the progress at that point and 
wtiai the review procedure will be. Also give the rc*sources used iince the last milestone. For 
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me of tyittm pvt th« cumni «• of tht tytum at ihai mtlMloiM. Each miintona hat 2 
i»nn<l(nca levels, one for time etitmain and one for resource eKpentllluies. For esllirated 
future nulettone, ihe Hni confidence level for the probability of reaching the mileilone at 
that date. The second It for the accuracy of the resources used. For past nulestonct. the 
first confidence level ia normally 1 00^ (actual date) while the second Is an estimate on the 
accuracy of the accounting system. 

M. DOCUMENTATION 

For each time of documentation developed, state the type of documentation, its purpose, 
the date it should be completed, its size and list any tools used ui its production. (At the 
beginning of the project these should be estimates, at the end of the project, they should be 
accurate figurea ) 

N PROBLEMS 

Give the three most difficult problems you eapect to encounter managing tlus project. 
Please be as specific as pouible. 

O. QUALITY ASSURA.NCE 

To what do you attribute your com'idence in the completed system. Be as specific as 
possible. 



OINIRAL PROJICT SUMMARY 


I mOJICT NAHM - - -- OATS 

A. FNOJICT OIICAirriOM 
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C TIMI 

IMTI OiM InA OIM iHtmliA Utmnw MMmi Otw 

Con»A«i«« U««t 

0. Cm 

Com • ■ Malimim AMiiaMa t — _ CaiA IA w* U«A __ 

How Con OoUrnOnoA 

Nneimoi: InnotioH 1/3Wiv tnvimt, CoioaIoHm 

ToM Awhh MomIm 

Otiin Cora: ComAMor rimo ihri) Ooomnoimtloii • ___^__ 


OUmtI I 0th* ( I 
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tin o< (vnoiti Wof At. Dau W*At _____ Intn o ononi 
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Olh* ( I 

Itfuciuta ol Svnam IChaah OimI: 

tlnAlaO«*tav 

— 0«*tav Iwuanna (Numh* of 0«*lav« — ,— Atf. IIm ___ ) 

__ InAapanAant ^o*an« INumb* of Atf. tlia___ 1 

Otfim Yoi»» Conaaoi of a MoAuta _______________________ 


Nun** of Madulai Aanfa in MoAula llta: Min ____ K*a. __^_ A«|.. 

Nun** af Oiff*am I/O Aoimau _____ 

lt»l il/MI 
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Othar 
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I. AUTOMATIO took Ulto 



I ONQANiZATiON 

Hm art tha tawiwiA Or«aniiari: 


Aotaai Avionnal: 



K. STANDARDS 


Tva* 

Ont tonal 

Raoulfad 

Tltlft of DaoufiMfli _ -- — 

Tvm 

OtttiOfMl 

AotM*kari 

Titia ef Doeumani 


Typa ____________________________ Optioiial ________ Raquirad 
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o 
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OR/G/W/ij, 
OF POO.) 


I. MliKTONU 

nwif — ■ - ... riMtm CanfMwMt l««tt 

How Ctotwmnotf ^———— 

Noviowon - - 

Rtponint fteooduro — — 

HttouTM bponditwa: Com — Pwoin Month* Comfuiw Tim* _____ hr*. , 

Hi* oI lyitom ___ CoMWtno* L*«*t ___— _ 

Phot* iMimoiod Dm* Con«d*no* L**M 

How OoiwnonMl ______________ ■ ______ 

h.. -tw«r« ■- - 

hiooftini Kootriw* ■ 

Rmowoo EiptndlluMi: OoM _____ >*TMn Month* Compiit** Tim* ... hn. ______ 

III* of lv*t*ffl ______ ConfUMM* L*«*l _____ 

(hi*** btlniM*h Dm* ConltdMi** L*»*l 

How O*t*rn0n*d - — > 

R**i*««*r* 

N*«ortint ^ooodur* — 

R**oufO* Eaponriltin**: Cow Parton Month* ____ Gomputot TInw ^___ hr*. ■ , . 

tif* of lyttom _____ Conf W*no* L*«*l 

Ph*iO OMliiMtit a««« ■ CanrWMM LiM 

How 0*t*rmin*d 

R«m*w*rt - - 

n*portln( frooodur* — 

R«uure* Ekpanditur**: CoM P*rton (Month* Computtr Tim* hr*. 

Sit* of lvM*m Conf id*no* L*»*f _____ 

Ptw* Eitlnwtod 0*t* ConfMon** L***f 

How Ottorminod 

R*«i*w*ri ___________________________________________ 

n«Oortln| Aseodur* ■■ ■ 

RoMure* Enponditur**; Com Porton (Month* Computor Tim* hr*. 

Sin of SvM*m ConfM*no* L*«*i 


nwt* 6irim*t*d 0*1* CoiMWane* L*«*l-. 

How 0*t*rmin*d - 

R*iii*w<rt .. . - 

Rtportino Praoodur* — _ 

R*aoure* Eapenditur**: Ob*i Paton Month* Computw Tim* hr*. 

SiMof Syi«*m Confid*ne* L*«*< _. 


Ph*j* ____________________________ Eitl"n*t*d Dm* ______________ Confid*ne* L**M 

How OM*rmin*d • - 

R*vi*w*n ■ ■ - - 

Rrportinq Aocwtur* 

Rwourc* Expoiditurw: CoM Pirion (Month* Computa Tim* hr*. 

Sill of Syn*m _______ Confid*ne* L«v*l _____^ 


Phil* Eitim*t*d 0«t« Confid«ne* L*«*l 

How 0*«*< Tilnwi -- 

R **■*»“'< - - - ■ 

Riporting Proctdur* 

Rc_iurc* E«p*nditur«*: Con Paion Month* Conputa ^im* hr*. 

Sit* of Sy*t«m ______ Confidtne* L*«*( 


1i<Wl wOOTirhuAllon 
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ORlGir-iA- r,/E IS 

OF POOR QUALITY 


M. OOCUMINTATION 
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___ fattmowl Sin 
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N. PNOtLtm 

I 

I Stati (hi thru moM dlHImilt pioWimi you mpmi to inaouiitir in gomptnint Ihi proiMt It • tnott dlffloulll 

I 

I t 


O. QUAUTY AUUMANCC 

sun thi thru rAO$t ‘npirant mptat§ of ihi duipn, diutopminl ind tutinp of tho lyitim to which you ittributc your 
eonfldinvi thi aompliicd lytlcnv It ■ meet imperunt) 

1 




i PERSON PILLING OJT FORM 

(i/fy) Coftitnih«ii«»n 
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Of poofi 


,v.rt iS 

qU/^*L\TV 


INSTRUCTIONS FOR COMFLITI JO THI RiSOURCt SUMMARY 

Thii form kcipi irKk ol thi pro|fcl coin on « wnkly bMK. It ihould tM llltid out by ih* projKi m«n«g«r ntty «n«k of the 
proitci du««iion. 

FROJSCT Oiv« proitci nimt. 

DATE. Liil dill form turntd in. 

NAME. N«ni« ol proitCt m«n« 4 «r 

WEEK OF. Lilt d«(t ol uch lucctiiivt Fridiy. 

MANPOWER. Lilt <11 ptrionnil on ihi pro|«ct on uptriti linn. Oiv* tn« numbtr ol houri «ach iprnt lliM wrtk on ihk pro|K> 

% OF MANAGEMENT. Add Iht % of timt !''•( pprton ippnl miniping thp pro|»;t during tbii rtporling ptriod. A ntw form ibould b« 
uMd if ihti \ cn«ng«t. 

COMPUTER USAQE Lid ill inKhinn uud on tht projtct For ucll micbiny giv* th« number of runt during Mch wnk «nd Iht 
tinouni of computer time uied 

OTHER. Lid eny other chergn to the protect. 
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RESOURCI 


M0J6CT 
NAME 


j WEEK OP 1 



MANPOWER (HOURS) ' i 

1 

1 


i ^ 

u ■ , 

i 

^ 


COMPUTER UoAv 

INO. nUNS/HOURS CMARGEOl 


t 


T 


OTHER CHARGES TO PROJECT 


I 



! 


J»0-1 


B 


I %OP 

' MGMT 





OF 


I . >£ 4S 

q'uauty 


INtTRUCTIONI FOR COMPLETINO TH| COMRONINT lUMMARV 

Thii forn'< i u(M lo kMP trKk at thi compontnii of i ivtum A comoontnt ii « oitct of tht lyitcm idtntifitd by ntm* or 
common function Ik.q.. in ^niry <n t iru cbtrt or boMlino diigrim for tho lyltom kt «ny oomt m timo. or t ihtrod lOCtion of data tucb 
41 4 COMMON bloek) With tn« informmon on thit form combmod witn tbo nformotion on tho Componont Statui Raport. tna struc 
tura and itilui of ttfa lytttm artd m dtvatopmant can bo monitorad. 

Thii form ittoufd ba f'ltad out for aacft componont it tha tima that tha componont >4 dafinad. at tha tima >1 t comptatad, and at 
any point ,n tima wnan a maior modification to tha componont a mada. It thould ba flllad out by tha parion raaponiibla for tha com 
ponant 

FROJICT. Oiva proiactiiamk. 

DATE. Giva data form flllad out. 

NAME OF COMFONENT. Giva noma lup to 8 characMrif by which tha componont will ba rafarrad to m othar forma. 

BRIEF DESCRIFTION. Stata function of eomponant. 

TYFE OF SOFTWARE. Chock all claiiificationi that apply. All common Wocki am r:>aarata eomponanti. 

STATUS OF COMFONENT. Chock whathar thii la a now component, whatnar it la a c irnponant undar davaiopmant la.g., a pravioua 
componont aummary hai already boon aubmittadl, or whathar thaeomporanl ,i now Cim.^nata 

A. CODE SFECIFICATIONS. Giva tha fori.i of daiign for this eomponant ;nd tall t.’ uno laval of datail tha apacificationa ara givan. 

FuneUonal— Componann ara dateribed ai a tat of functijnt, ai!Ch componont par.drming a canain action. 

Praaadural-Com'ionantt ara apacifiad in aoma algorithmic manner (a.g., uiing a FUU . 

Engliih— Componann ara apacifiad uting an Engliih Languaga prota icatemant of tha proMam. 

Formal-Sema othar formal lytfain it uiad to apacify tha componar-n. 

Ralativa to tha ana davalop -j" tha eomponant, rata tha praciiion of tha aoeeificationt. Vary praciu maant that no additional analyait 
on tha problem it naadad, pr .to meant that only aaay or trivial idaat have ' ca dav'lopad. and impracita maani that much work ttill 
ramaini in davaloping thit eomponant and iti banc ttructura. 

B. INTERFACES 

Give tna ralativa potition of thit eomponant m tha tyttam. Give tha number and Mat tha namaa of all eomponantt that call thit 
eomponant. and are callad by thit component. Alio, give tha names of eny componann or othar itami this eomponant tharat with 
othar eomponantt Ic.y.. COMMON blockt. axtarnal data). Tha eomponanti directly datcandad from thit component rafara to tha tree 
chan cr tha lyRam. If tna intarfacai ara not yet complata. ehack "Not Fully Spacifiad" 

C. PROGRAMMINQ LANGUAGES 

Lilt languagai lor aiwmbiy languages) to ba uiad to implement this component, if more than ona, lilt pareanragaa of each lln 
lines of aourea code). If there ara any cannraintt on tha component (a.g., am, akacution time) list them. Also give attimatad ana of 
finiahad eomponant in terms of aourea atitamants. (aitimata tiia w.th commanta and without eommantsi and resulting machine lan- 
guages lineluding data areas, but not COMMON blockti. 

Uaaful llami From Similar ftojaets 

t. bit previous components and proiacta which contribute various atpaets to this eomponant. 

2. For each such component, give tha percent of each of tha three luted aioecti it makes up la.g., a component may ba 50% of 
design but only 25% of coda due to changed mtarfaeas, ate.l. 

3. For each of tha three listed aspects, check what lavF of modifications are naeassary. 
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0. COr^KIXITY 

n*M your Mlt*> >n m« compMuity of tht implorntnuuon Alio 4PU0«»yi«t« t -i numbfr luy '^1 0* tii»]prn*nt tvp* itiltmanu 
(input lunmtoii «r« mcludod) <nd control itaumonii iihoM tn«t iinr th« 'low of control. • 9. iF, CALI GOTOl Thy ;um of thyi* 
two may not tM lOOfb It 9. CONTINUE. DIMENSION and REAL ilatamtnti will not Ot counttd) I 0 and dtdaraiioni ihouid Dt 
iiiHd M dinar 

t. RIIOURCCt TO IMKEMENT 

For tKh of thf ihrtt liittd phitti lOttiQn, Cod*. Ttttl. tnimatt eompuitr runt, timt nttdtri, noun to u. ''itmtnl, and tilh 
matad compittion data. If not known, or no tatimtia can ba givan. writa "unknown". 

F OmaiN OF COMfONCNT 

If thii eotnpontnt >t indapandcnt of any othtr comgonam of tha lyitam la.g.. n a low laval componani which n dtiignad firit. or 
II tha root noda of tha traa chan) than chack yaa, oiharwita chaek no 

If no II chackad, than expltin why tha componant wai addad. (Uiually only ona raaion will ba cnacktd. although mora may ba 
ehackad, if appropnalal. 

A lowar laaal tlakioratkin of a highar ia«al componant maani that an aaining componant waa aipandad to includa naw compo- 
nantt la.g.. axpanding traa charti L111 tha highar laval componant tima. 

Addad ai a drivar oa Intarfaca maam mat a calling program wai addad to call ainting componami. Liit thaaa eallad comronantt. 
A radaaign of an axiating componant maani that naw capabilitiai wara addad to an alraady aaiiting componant Wnta in nama. 

A ranaming of an oldar componant. Giva tha eld nama. 

A ragrouping of axiiting mataritl maani that layaral eomponanti wtia radaiignad with a naw componant raiulting from thn it- 
daugn. Ova tha old componant namai. 

Typa of addition. Why wai thn componant addad to tha ivitam at thn tima? Chack tha appropnatt raaion. INormaily. only 
ona ihould ba chackad, although mora can ba If approprlata.) 

Q. ADDITIONAL COMMENTS. AdJ any othar commann that will help akpiain tha purpoaa. daiign. and complaaity of thn com' 
ponant 

H. FCR 90 N RESPONSIBLE. Includa nama of parion laipondbla for implamanting componant. 

1. PERSON FILLING OUT FORM. GIva nama of pcrion filling out form. Thn normally 11 tha tama nama ai in H. 
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or- PUw'i 


COM^NENT SUMMARY 


)»flOjECT 

NAME OF COMPONENT. 

brief description 


DATE 

CREATION DATE. 


STATUS OF COMPONENT NEW 

TYPE OF SOFTWARE (OwU All TNit Aoplyl 

i'f> ProoiHint 
_AlfaritMnic 
I "9*^ Control 

A. CODE SPECIFICATIONS IChock Ail That Apply) 


UNDER OEVEL. 


. COMPLETED. 


.Syntmi Rtiatfd 
.DATA/COMMON Slock 
.Other 


LEVEL OF detail 


FORM or DESIGN 


I Component 
-I- 


Suboomponent 


T 


■uic Slock 

Segment 


Stmt Other 


Functional 


Procedural 


Engllih 


Formal 


, Otner I 


Praciiion ol Coda Soecilication V^ry Precita. 


. Preciie . 


.Impreclie . 


INTERFACES 

Numoar Componaiiii Callad. 


.Nama». 


.Not Fully Specilied 


Number Calling Thli Component . 


.Namet. 


.Not Fully Specitied . 


Number Shared Itemi. 


.Namet. 


.Not Fully Specified 


Number or Componenti Directly Deicended from Thu Component . 


. Namet . 


.Not Fully Soecifled 


PROGRAMMING LANGUAGES 
Languaget Uted and Parcontegei. 


CONSTRAINT PROSLEM EXPECTED 


Conttramt 

Preient 


Componant Meets 
Conttramt 


' Memory Space 


I Etecution Time 
(Jtner I 


I 


Site Source Statements lInciuO-ng Commantil 

Source Statements iNot Including CommenttI 
Utelul Items From Similar Proiecti 


Machine Sytet. 


Soec'ication 


Comoonent 


Prpiact 


Design 


Coot 


\ Maior Minor Nona Ma|or Minor None \ Maior Minor None 
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> ' • .1 

OF FGO.< Qw • . i V 


0 COMTLfXITY 


ComoMiitv of Function Etiy . 

- % Auiynmani _ 


. Modarait- 


‘ . 


. '• Control Stjtamtnti . 


1 Ottwr Stllt.ia>''> 1*4. 0 «tt Ctci 




I 


E nCSOURCES TO IMPLEMENT 



Run! 

Computer Time (mini 

Effort (hrti 

Est Cdmpittion Cac« 




i 

1 Ten ; ! 

1 1 


n It tftii component <ndtO«nd«nt o( th« Mining rr»mnn«MH»> v«. , io 

!• No dwerib* mlation of thii eamponant to in* tuning (ytnm 

.nt*rtM <1 a lower level eltboretion of righer level compontnii nemeii 

aOOeO *1 e driver or mierfec* for Mining componenti lntrTt*il__ 

j reUtngn Ito add new caotbilitv) of eaiiimg component! nam*i|__ 

1 rtneming of eximng component |nam*l__ 

___'tgrouping of tuning mettritl from itvtrti oomponenti lnam*il__ 


ninar __________________________ 

Type of Addition 

trror correction 

___ planned enhancement 

iinpiementation of requirement! change 
___ improvement of clarity, maintaineoiliiy. or docuirwntttion 
oti *r !*« plain below I 

G ADDITIONAL COMMENTS 


improvement of u!*r Mrvie* 

utility for development purpoMi only 

ontimifttlan of timt/|ptC*/*OCurtCV 
adaptation -i tnvironmerii cnangi 


'm PCH^niu PfcPQMctRi f rnR impi ementinc. component 

!i PERSON Filling oi-T form 

^ I 

)90 ) f«r 
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OR:r.i:v^'- r; c i 

OF POOR QUALI7Y 


INITRUCTIONI FON COM»L(TINO TMt COMKJMNT STATUt NITONT 


T>)ii form It to M uMtt 10 tccuriWy kttp lr«eh ot Ui« dtvt(aom«nt of «tcn oompononi in ifto tyttwn A Comoononi Summory 
Mtoon ifiould •■lit fo' toen eomponom montionod. Th* form n to bo (urnod in m mo ond of ooen mrook Motto fill out oithtt dotty 
or onco ooeh wook If doily. Ifion t 9 »von comoononi mty bo littod lovoril limot during iho i»urto of ■ wook For otcfi comoononi 
iitl (ho numbof of bouii iponl on ooeb of tho littod octivilitt. Thtt lorm mould bo fillod out by oortont workinf on mo protoet 

^NOJtCT Momo of mo proioci 

PNOOHAMMtN Nomt ol profrommor 

DATE Oolo report lurntd m Utuolly mo doto of • Fridoy 

COMMININI Nome of componont. Elthor i port of mo tytttm itructuro for wftich tboro it i component uimmory form or ono of 
the tollowinb 

JCL. Oovolopint commend linfuobo inttructiont 
Okortoy. Oovnlopinf tyttom ovority ttrueturo 
Utor Outdo. Utor'i Ouido Oocumtntotion 
tytMm Ooloripitan. Sytiom Ootcription Documtnution. 

OEtlON 

CrMto. W'ltinf of o componont dttign 

Aood Aoodinb Iby poor) of dotign to look for trrert It.g., poor roviowl 

Formol Royiow. Formel mooting of lovortl individuoli for purpoto of okplom ng dotign Alto meludo time tponi m proporing for 
rtviow All mow attending remew tfiould lilt eomponenti dlKuuod m tnoir own Component Stotui Aeport for mot week. 

COOE/OIVELOdMENT 

Code. Writing executable inttructiont and daik chocking program. 

Atod. Code reading by peer Similar to Ooiign Read above. 

Format Review. Review of coded eomponenti. Smilar to Oiitign Review above 
TESTINQ 

Unit Unit letting. Tett run with toit data on linglo modulo. 

Intog Integration t'ttingof laveral eomponenti. 

Review. Review of tatting ttatui. 

OTHER. Anv other aipect ralatad to a component of the protect not aireadv Mvarod other chan Oeugn Coda Oevelopmant Tett 
la g Documentation of a loecific component). L.it tvpa of activity, and houri ipent on that activity A let of activitiei hat t>een 
'litep for which time may be charr^ed to the overall proiact 

Travel Time ipeni on official travel related to thit protect. uncluOing tripi to and 'rom GSFCl 

Forim Tima ipeni on filling out reporting forrtn. 

Mootingi. Time loent m maatingi wrnen are not deiign or code review moatingi. 

Training. Training activitiet identified for protect 
Ace Ten. Acceptance Tatting activitiet. 




i 


OF pgoh yv- ^ ■/ 


COMTONINT STATUS RIPOAT 


^BOJir.T OATI 

MOONAMMiN 





ORIGINAL PA?" .'O 
OF PCC,r, 0 . V 


iNtTNUCriONt »ON OOMPIITINO THI COMTUTIN MOONAM HUN ANAlVtll raWM 

Thii form will Da uMO (o moniiiH m* tcti«ii«i for wiiicn ifia oomouitr 't uMd m (fw oauria of • orotfci iif« cveit An tniry ineuu 
M inoot for MCfi oamoulir 'urt-niclgOinf «ll (ctwilitt porformoO wf>on tho oomwlor i uMO <n mtorMtivO irooo 

moORAMMIR iWnlt JOwn n«m« of uorten Oroaarino comewitr runt. Tf«ii may n«i nocoManly M in* imtan runmnf int oro«r«m 
ii • iiOrtritnl 

mOJICT iV' (• down (irowci n«m*. Ui( • d>ff*r*ni form for ««cn nroi*ei 

COMRUT' > .«*!• in* mtcnm* an wnicn in*i* 'uni w«r* m«dt i* S< 3t0 ' £11 

OATI 0\,» form luinao n 

JOt lO id*nlifiC«lion of lOD 

NUN OATI. 0*<* 'un luOmilliO n form*! MM-00 ifnonifvo«yl 
INTINACTIVI. f*'*ca in X f lf«* >un wM i>'‘:'nin*0 from «n mtirictiv* lirmuiti 
NUN ^UNNOU *ltc* an X m all Ooaac that dawiba inn run 

Umi Taat A our com of ifi* run ii lo tati ona or mar* camponanli without ih* r*it of ih* lyitrm bainf oonfifurad into ih* >oad 
n«dui* A 'un which uiai a tatt onvar' wouV fall into thit catofory. 

Syitatn Taat Thu run aaacuttt a loab mooui* which oontaina all of in* euriantiy avciiabia ivitam in oroar lo tail on* or .nor* 
oamivinanii in a full lyitam configiaanon 

•anahmark Taav Th.i .| a raoartifioaiion lyp* run A run in*i hat Miceaiafuiiy aaacuiad in* pan 'l now rerun to y*nfy ihat 
cartain cauabilmaa itiil anil 

Maimanarwa/Utilltv. A purooi* of inn run ii lo car form * iibrary-iyu*' function. Ciamplaa ar* runt mat update nurc*. oraat* 
backupa. dalat* conipraiiyoopy data Mti. 

Comp il e/ A tiatnbly/ Link. A uuipoi* of fha 'un .i lo cnack for arrori m |h* oompn* aiiamoly and or link ttaoi. A run which .n. 
ciudai on* or moi* of tn*M rtapt iimpiy at i praraqunit* to a lyttam *k*cution yyouM not 'all >nio ihii catavory 

Oabup Nun. Tn.i run wai lubmntao n order to nyatt.gai* a known arror. 

Olhar. Thii run n*i j purpoM whipn rjoat not f*ii .nio on* of Ih* othar e*t*«or>*i. faampiat ar* runt which accat* oinar lyitamt 
n ordar to aid .n in* oaiian dy**K>pm*ni and'or laatmp of ih* proiaa under itudy 

COMhONfNTS OP INTINIST. U h ail compananti mppriani to ihii run (a.p., oomponanit bamp tailed compiled, copied ate 1 

PINST NUN. h'ac* an X ri*r* .| ihii .i ;h* inn urn* iny of th* i.ited componantt n*y* aaan oroctiiad by Ih* compuiar 'or in* our- 
001* of 'un ipactfiad 

MIKTt OBJiCTIVIt. thii .i * luOian y* avaiuanon of wnathar ih* run latnfiad your obiaclivat Nuni ihai tarminat* .n trrori may o» 
laiii'actory .f Ih* obiactiv* wai to ocai* arrori or to tail for corractnaii. rum inai larminai* normally may b* unwiiifaciorv f tn* pur 
COM wai to local* an arror known ;i> 0* praiam Thui ihii guaatiOn .i muapandani of whathar in* propram coniainad any arron or not 

NUN NKULTl Check tn* tMi inai bail aaicriOai in* ratuRi of inn run Normally only on* boa a chaekad. allhoupn more tn*r) on* 
may o* cnackan f auuroorial* 

Good Nun Pyo<(ram ran to larmmaiion with no known arrori. 

Satup Irror (.rrer in .yaalinp propram deck 

i,ubmii Error Deck lubrmiita ncorractly. 'awureay un***iiabl*. ktvpunch arror or pantral lubmiiaion error 
JCL Error JCL italamant noorracl IJCL earot mittypad ihouid b* iiilad under lubmit trrori.1 

.'iiKw Setup Error Suen at .niuff ciant loaca or i.mt loac fitd 'or iob nap Thii ifyokki not b* cauiad by propram ar'or 
Machin* Error. Errort outia k t tn* ooniroi of tn* proprammir 
HarOwar* Error Maenm* malfunction 

SoUwai* Error Syllam cram or lyitam propram arroi i*.p,, error n EORTNAN compiiari 
hropiam Error. Error cauiao by n* lupmuiao propram 

Compii* Error Th* lou'c* propram coniaini an error wnicn i found oy tn* compiiaf or jliamOlar 
Link Error Th* loadar or linkage addor 'indi an error, 

Ekacut* Error Sylian' error mtiiapai ar* panaraiad durmp in* aaacution itao oombiy cauimg an abend 

Uir Oanaraiad Error '^n* proijram larmmaiai n a proprammr p*<y*rai*4 error manapt wnicn t not a lyitam error 

bin 10 Compi in Ti * propram larminaiad win no error maiiap* nowavar, in# laiultl ir# nonrr*B | pnifyinp inji ina.* . 
jomain.nq v. -np ydn tha proprari 

COMMENTS. '* rou oaiiav* mat ,our aniwari to tn*i* tjuailioni do "oi Jdaouaialy i.-naractar t* mi -un you mjy add joy jodilonai 
oommanii mat you w.ih Aiio uia m.i loact lo ndicat* ' me 'un wai on balora you riao a cnanct lo avainat* 'aiuiti 
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ORlGlM.V^ 13 

OF pocn QL'AirrY 


IMtTRUCroiM rOR COMRt.IT. 3 TMf O4AN0I MPORT roNM 


TIm totmit UM4 10 Moo tfoch 0 * 411 cAonfN moOo to 4 iviiom a enonfl '* ***0 OtIirHiOA lo too O04i«n Ootumoniit ion. <v ceOo 
fonoroMO for 4 proioot losA chonfo con io mouffif of m o itoo m ih# orocoM of tronoforminf ifio onoinof Mfnvoro (M4.4n into 4 com 
oiott workinf lyitom Tht iniiiol crooiion of MClionc of frool* coOt or d04ir> ii nof 4 cftonfi 

Ono enonfo rooort form otoulO ho fiHoOool for toch chonfo WfMro lovoroi enonfoi oro moOo iimulionoouov fnr ciifforoni roo' 
tern 4 Mooroto form ifiouiO bo comototoO for ooofi rooron 

NUMMN. A uniquo .donitfior cor 'orm oor oov con4iMir«| nf nidoli fellotrod hv 4 toquonco numbor Tho inmoii tnouUl bo mo40 o' 
trio eorion fillino owl ibo form Tho toquonco numbor mould bo 4 eotidvo inrofn inqieonno mo number of formi f*MoO out to 'or ilur 
ino I'M dov Mumbor OMWOf 'nqtcoiM tho firii form of tho doy 'Mlod out by OMW 0MW03 il Iho looend form ihof doy tte. 

MOJICT NAMC. Trto nomo of tho doyoiobmoni protoci 

CURNINT OATI. Tho dolo on which $n tntry ii firti modo on iho form, oyrm if tho form it net complotod on thoi doy 

MCTION A-IOINrtRf<';ATION 


MAION. lopioin why tho chonbo it bomq mode. 

OltCNIRTIOM. Ooieribo mo ehonio mot ii bo<n« mode. Thq mould net h« on tho vonoWo nomo or bit lovol. but mould bo tuffi 
Citntly tbotroot 40 Ihoi tho function of tho Chonqod cedo eon bO dotorminoid. o.|., "tho input buffor wot cloorod." rtihor thon "trroy 
buff wol tot to wro " 

IRPICT. Whol cempenonit lor documontti oro chonbodf Uit tho nomoi of t" eomponontt ond detumonti modified ot pon of thy 
chonfO includinq voriion numport 

IPPONT: Whoi oddllionol oompononti (nr documontil wore utminod >n dotormininq whot ehonfo wot noodod> litt til cemoononti 
tnd doeumonti thot wore ooominoO. but wore not tctuoily chtnfod. n doeidino whot cnonpo to mtlio. how to mofco it. tnd where to 
mtko It Thit lilt ihouid not ovorloo with tho utt of compononti tnd doeumontt tctuoily ehondod. 

OATH OR CHANOI. Need for cfionpo dttornnnod on Give tho dote on which it wot firii rooluod mot t chtnot wti noodod 
Ottndo iiortod on Qiyo the dote on which tho chonfo wot itonod. 

Whot wot mo effort in portort-timo required to undoritond tnd imMomont tht chonqt' 

Qivo th« boil oyoiitbio oitimoto of tho totti time noodod to undoritond whot chineo htd to bo mode tnd how to moke it. nclud 
tn% tht implomentoliar) time Thu mould include Iho time of ill portoni mvouid m mtkinq the chonfo At on oiomplo. if two ooopit 
toen wi’.rkod 9 hourt on the ehonqi, the ipoeo mirkod "one doy to 3 do/t" mould bo chockod 

tICTtON l-rvpl OR CHANOI 

Chock the one boa thot bttt dotenboi tho chonqo. If none of the chonqt doicriptiont toom to 'll. chock olhor tnd give i dotiilod 
dttcription of Iho ehonqt m Section E I' leverti o' thi doKriptioni teem equoily ippropritte more then one boa mty be checked 

Error Correotlon > chenge medi to correct m error m previoul work. If Ihit boa t cheeked Scctiont C ind 0 o' the change report 
form mould be completed. 

RItnned Enhoncoment. The ntertion o' i body o' code mto t program ituO tntt wot irnt tiiy created at a dummy 'or ten ng purooiii 
or idding ctotihlity to an iiretdy Kanimg component h pan o' a planned ncrementai development 

Impiemenitnon of Roquironieoa Chonge Altering the tyttem to conform to a change m rtouirementt impoied by the cuitomer 

Improvement of Clority. Uatntpnobtllfy. or ODOumonttiion Changei medo to mprove code quality, luch at morcvmg ndentanon o' 
coda 'otoduonemg i iboii 'or 'oadtOiiiiy adding or updating documentation or correcting literary errort m t luopraiting radundart 
information or replacing multiply-occurring teciioni o' coda with proetdura cam Corrtctiom o' yioittioni o' programming ttandardi 
tnd dtiign 'mprovememi mat should heva been vitiWe m the 'unetionei looci'icai ons o' componenti o' the system tre to be treated 
ei error co^rectiont. Documentation uodatat mada Cdncemiianiiy .vith a ehangt ihouid be tretteu as a pan o' mat change im] ciessi 
'led with the primary ctutt o' the change 

Improyement o' Utor Seryicet. Dating lysttm aeya'opm*ni nn.v.oual programmers mjv <md that n-m .ery ' *t'« »«t’s lOrv ■"ev ci" 
provide i»w user .vitn eddit onel 'ec ' ties o" too o' me 'u"ct onai 'equ'remenu o< me system Sucn changes I'r nitstcl is mpnv* 
menu to user servicet 
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lnMRWn/D«M(tm ol 0«kii« Cotf*. Chan«« m«d« to iiw proirwn itat lOKifietllv to proviOt tOd>tion*i information (furmi t«t rum lo 
that trrort can ba itoiatad. 

OauiMta Tlma/loaaa/Aaavraay. An ooumitatton >• a 'oeaMiad adluatmant of t*w oroaram wftoM main purpow i> to raOuea m a»««u- 
I >n iina or mamory rao<:>ramana, Of to oOtain rawto of araatar numarieal acnracv by lunini ma aiaortthmt uiaO to tba tpaeifie 
proUtm batna loivoO. 

Adaptttlon fa (mrlroMfiain Cbanfa. Tha "boundarv" of a toffwara tyitam ii Oafinad to meiuda lutt thoia prearami wtioaa davaiop* 
mant and maintananca ia baina monitorad u Pan of tha lofMara anainaarina laboratory profact. A chana# iwhoaa eauia Hat ouuida 
thit boundary (a.a.. m ratoonia to an oparailna lyitam, compilar . or hardwara ehanaai •• raac'dad at tnviranmantally eauitd. 

Wat mora than ona eompoi«ani affaetad by tha ehanaaf A oomponani n dafinad to tm Miraetiy invoivad in a chanpa if it eontaim 
lubroutinai that art chtnaad and it eontaim no tubcomponantt oontaininy thoto lubroutinaa. Oack yat if tha ehanga diractiy invoivat 
mora than ona componant of tha tyttam, no otharwita. It may ba tha eaaa that a e'lanst tn ona lubroutina/eomoonant anil raouira 
toma 'utura adluatmant in othar eomponanu Ithata oomponanti may not avan hata baan oodad yat, or thair adaptation may ba pon- 
ponadj. In luch caaaa, tha affaen of tha ehanfa invotva mora than ona eomponant avan though only ona modula waa noted at changad 
on ;nit form. 


SICTION C>TVH OI> iNNOh 

Otack tha ona boa that boat datcribaa tha arror. If nona of tha arror datorlptiom taam to fit. Jiack othar and giva a dauilad da- 
HTlption of tha arror In Section C. 

Requirenianta Inaarran or Mlaintarpreted. Requirementt may ba Incorract (incomitteni or ambiguouti, or their meaning may be mit- 
interpreted. In either eaia. an arror of thia type, if undetected eerly, may propagete through deaiqn and into coda. Z'ltn If undaiaetad 
until accepunee taeting (or maintenance), erron roaultlng from incorrect or miiinterpretad raauiremanta ihould ba elaaiified in tha re- 
quiremantf arror eatagory. 

Functional Ipadflaatlom Ineorraat or Mliintupi -tad. Functional lorclfteatlom are takan to be a waeificaiion of a component u a tat 
of functioni defining the output for any input. Si- ilar to requirame>m, tpaeifleationt may ba elthar incorract or mliintarpretad. Er- 
rert in the ipeeificatiom that occur aa a reault of mi jndaritandinga of raquiremanti ara claMiflad at miiimtrpreted raquiramenti arron 
and not incorract ipacificatiom. Specification erroi that mult from miiunderttar^dingi among thota writing the ipeeificatiom are 
etauified at incorrect ipeeificatiom. Crroif in coda jr daiign or documenti raiulting from moorreet or mliinitrort'iad ipacificatiom 
Ihould be dauified in the loaeifieatiom error category. 

Detign Error Ineoleing Sayeial Componaiwt. A deiign deciiion ii a choice of c"ianiution of a eomponant into lubcomeonenti. in- 
cluding tha ipeeif leation of tha interfacei among the Mbeomponenti. A daiign arror it a daiign deciiion that raiulti in om of the fol- 
lowing: 

a intarfKn that contain intuffleiem. unnaceaaarv, or radundam information; 

• a let of lubcomponenn that do not laiiify tha ipeeifietiioni of tha component (i.e., one or more of tha lubcomponenti do 
not have the ctpabilitiaa needed lo Mtiify the uia intended for the component). 

fiMicMa^^laiign error may remit from incorrect or miiinterpretad requirement! or ipeeif ieat*;ni. In luch eaier. the error 
ktiouid not ba ctait <ied ai a deiign error, but m a requirement! or tpaeification error. 

Error In the Qaiign or Implamenution of a tingle Componem. Won iimple, localited programming miiukei fall into thli category. It 
carMng ttoia caiet where the u. yaniiaiion of the lynam into eomponanu and their mierfacai it correct, but a particular eomponant 
dcH net behave according to m intended uia (i.e.. doai not eorreipond to in ipeeification) Thu ma occur becauie the algorithm 
uttd m caiigning the component ii incorrect, or beeauta the implementation of the algorithm ii incorrect. If the algorithm hat a writ- 
tin tpeciflcaiion prior to code generation, mo tha tpaeification u incorrect or mitmtarpreted, the error ii not claiiified ai a dHign or 
mpiemintation error, but ai a ipeeification error. If the erroneoui ilgorithm hai no written ipeeification. or If the implementation of 
tha algorithm hai errori not attributable :o any othar category, then the error ii dauified u an error m the daiign or implementation 
of a tingle comoonent. 

Miiundentanding of Eatarnal Environment, Except Language. Check thii box if the error reiuited from miiuken auumptioni about 
tha hardware or toftware environment m which tha program ooaratei (i.e.. that loftware ouuide the "tioundarv'' .if the project -tee 
"adaptation to enviror.ment change" >n Section SI. Included here are miitaken auumptioni about how the operating lyitem worki, 
aoout how the hardware >i controlled, about reiponie of peripneraii to varioul command!, about the operation of tha library tyitem, 
about the interface to ipecial diipiav hardware or loftware, «tc. 

Error in Die of frogramming Languege/Compller. Erron m the uie of the language/eompiler are thoie erron that remit from lome 
miiundentanding of how the compiler worki, how the language provided run-time luppon tyitem operatei, or tome miiundentanding 
of particular language featurei. Not included m thii category are clerical errori le.g., typoil that lead to compilation errori. 
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OwiMt tim. Q«ric«l ut itioM mtoit inti occur m m« mccAtnicci trtntIMiorr o* «n itam )rom on# format to anol^ar ft.« . one 
coOinti (heal to anolharl. o' from ona mtdfum to anotfiar (0 9 . codtnf inaaii to cardll No irtiarpraitlion or lamaniic irantlalion it m 
•olvad III lucti a procaw. 

fON OIIION ON l«INLIMINTATlON INNONt ONtV 

Dili taction lAoiild ba fillaO out only if ina error ««im f daaifn error, meolving layaraf comoonrntt. or if it wai an error in the da- 
tion 01 imnlainaiitaiion of a nnfla comeonani. Error* that occur n' iha datign of a lyitam. luMvi'am tat of componenie. or tmgia 
component or in Iha impfatnentation of a im9le compo lant. may ta. catavoniad m ona of two wavi Either mere wai an arror n the 
uia of data, or there wat an arror m the function of a component (luch at an al9orithmic or computational errnr laeulting m jrogram 
nahavior not corraaponding to the iniandad uta of tha utograml Data uia arror* can da chartciariitd at either incorrect yclua* lot 
data itami or improper aeiumptioni about Iha iiructura of data itami le.g.. array iiaat or dimaniioni, or ordering of iiam* ir. a lutl 
Errort myolving the function of a cemponant iiKluda control and computational error*, luch a* incorrect laquancing of itatan'ent*. 
omitted iiatamant* (where tuch are not clerical error*), improperly computed aapraaaion*, omitted capabiliua* of the componant;ii. etc. 

IICTION O-VALIOATION AND NBhAIN 

What teata Iha aatiyttfaa uead to vaildeta the program, *e detaai the arror, and find It* aauief 

The purpo*a o* thi* taction it to d.icover how it becama known that an arror aiialing and how tha caute of the arror wat datar 
mined. A check thould ba pul m the flrti column for each method utad for yalldating the componantitl where the error wat found A 
check thould ba put in the tacond column on the tame line a* tha method by which the tymptom* of ihit particular error wat flrtt 
noiad The third arid fourth column* rater to activitia* utad to find the cau** of the error, one* it wat known that the error aaittad. 

In the third column, check all tcchnioue* utad m trying to find Iha cauta of the error In the fourth column, check ihoaa technique* 
that yielded the information naadad to find tha cau**. in tom* cat**, tuch a* tom* error* found by cod* reading, ih* tachniqualt) utad 
to find tn* error and diioovar It* cauta will b* the tarn*. N '* that arror maetagHhav* been dividad into two caiagori**: ihotapre 
ciuctd by tha luppori tyttam la.g., compilar, nparaiing tvttam), and Ihota detignad into Ih* cod* for th* ipaciflc purpota* of Ih* pro|' 
ect Telling hat alto bean divided into two catagori**: tatl run* mad* prior to accaptanc* tatting Ipra-accaptanc* tatt runil. and ac 
captanea tain If activitia* other than ihoa* iiitad m the table war* utad in finding th* arror or ditcovaring itt cauta, ehack oihar m th* 
appropriat* column, and datcrib* th* actiyiiiat utad in Section E. Thit tabi* inavitably ha* tom* redundancy a check in column 2 
mutt alwayi have a corratponding check m column I, iimilarly with eolumnt 4 and 3 

What wat th* time utad to iteiate the oautaf 

Check Ih* ipac* that moat dotaly tpproiimelai th* tim* required to iiolat* th* cauta of th* error Thii thould b* th* total of 
th* iim* that wat ipani m th* actiyitie* triad to fmd th* cauta If th* cauta 01 th* arror wa* navar found, and a workaround war utad 
check th* appropriate bon. If the cauta wat navar found and a workaround wat not utad. aeplam the circumitancet <n Section E 

Wa* thi* arror ralatad to a prayloui ehanga? 

Chang** to toftwar* may retult m errort bacaut* of onr : mor* of lavaral raaiont 

• th* Chang* wa* incorrectly implamanted, 1.*.. did not conform to itt tpacification; 

a th* Chang* invalidated an attumption mad* eltawhar* m th* toftwar*. 

a an ataumption mad* about th* ratt of th* toftwar* in th* datign of the change wa* incorrect 

An error i* ralatad to a previout Chang* if it retult* from on* of th* above thra* condition*. Error* that are uncovered by change*. 

I * . tn error matked by tnothar that it ravaalad whan th* latter it oorractad. do not belong in thi* category It th* arror 11 related to a 
oreviout Chang*, giv* th* number and data of the change rapon form of th* ralatad Chang*. Whan did th* error enter th* tyttam’ 

Check th* t>o» that matt dotaly rapratantt th* phtt* m th* arroraout componanii’ davaloomant in which rhe error wat mnoducad 


SECTION E-AOOITIONAL INFONMATION 

Thii taction ■* intended to parmii further eapienation of any nami you faat may b* iignificant m cattgonting tha ehanga im. 
eluding arror corractiont) It Ih* “oihar" caiagory wat crtackad in any of th* praviou* seclioni of th* form, a fullar teplanauon thould 
be given her* Do not hetitat* to give a full daeenption of th* error or cheng* or any doubtt you may have m claiiifying it Th* ac 
curacy of out analytil n dapendant on tha amount and accuracy of tha data you orovida for u* Tha itudv w* art performing .t an at 
tempt to do a careful, datailad mvatiigaiion of tha firae**i*i that go on during toftwar* davalopmant. th* kmdi of changai and arror* 
that occur during davalopmant. and th* raaaoni for thair occurranc*. With your halo, w* hoo* to gam enough mtight into the detign 
coding, and tetting of programt 10 that proootad taehniquat for coping with loftwart changei and raduemg me number cr error* can oa 
evalualad Your cooperation and paiianct m comoiaiing tha change report form each tima you maka a ehanga to a documant or oro 
art naadad and toprtciatad. 
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0KI6IMAL PAv:E S3 

OF FOCn QUALITY 


NUMtIR , 


CHANOf REPORT PORM 


MOJECT NAME. 


CURRENT OATS . 



SSCTtON A . lOENTIEICATION 


REASON Why wa th* ch«n«i 


DESCRIPTION Whti cntnfi WM matt?. 


EPPECT Whti comDOfMflti (or doourntnnl rra oAanfid? (Ineluda yartionl. 


EPPORT What todltlontl oomoontnti (or wtra tatminad In dawmininf wAat oRan(i wat ntadtd?. 


(Month Pay Vaa> 


Natd for ohtnta daarmlnad on . 
Chanfi narad on 



Wfiai wat tha aftort in oaraon iima rtquind to undantand and implamani tha ehanft? 


.1 hour or lata. I hour to I day. 


.1 day to 3 dava. 


.mora dun 3 dava 


SECTION B • TYPE OF CHANGE (How ia thia ctiangt baat ehartcttrltadPI 


Error corranion 


□ Plannad anhancamam 


C Implamantaiion of raquiramanti ehanoa □ Adaotation 

□ Improvamant of Jtrlty. mtinitintbllliv. or documanution □ Othar (Eipl 

C Imorovamant of uatr atryicaa 

Wat mora than ona oomponant tHr/cttd by tha ehanft? Ym No — 


FOR ERROR CORRECTIONS ONLY 
SECTION C • TYPE OP ERROR (ffow la ihia error beat eharatarliad?l 


□ Inaattion/dalatlon of dthuf oodt 

□ Ootimluiion of tima/tMatfaaeuracv 

□ Adaotation to anylronmant ehan«a 

□ Othar (Eiplain In E) 


□ Raqulramanta irteorraet or miaintarpratad 

□ Puneiionti tpaciflcaiiont inoorraa or miaintarpratad 
Oaiign error, involvmt tarartl componantt 


□ MIturalartttndInt of aitamtl anyirotHnant, aaotot lanfuape 

□ Error In uta of profrtmmlnf languafa/oompilar 

□ C'arleal trror 


Error in tha datign or impiamtnitiion of a iingla oomponani □ Othar (Explain in E) 


FOR OSPtdN OR IMPLEMENTATION ERRORS rWLV 
PIf tha error wta in datign or implamtntatlon: 

Tha error wat a mittakan auumptlon about tha value or ttnietura of dtu 


Tha error wat a mifiake m control logic or computation of an axpiettlon. 







ORIGlimi PAGE 13 
OF POOR QUALITY 


ftm MRON COMMCnONt ONLV 
ilCnON 0 • validation and NirAIN 
Whil «Mn um4 10 vtUtfMi Vw program. wtMt tht •rror, «i4 flno id eiuitl 



AaPyllia* 
Utad for 
Pfotram 
VaUdadon 

AcOviitat 

liiaaawfui 
In OataatMt 
Error Symptomi 

AathrliMa 
Triad to 
Find 
Caua 

AotNItla* 

luaaaiaful 

In Plitdlni 1 

Cauaa 

Fr*>aaeaptano* oait >una 







bhbhhhhb 

BHHVHHIH 
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IHHHHlHi 



Cod* taadiitt by protrvnmar 






; 

t 




, T«lkt Mfift om$r orBfraffmn 


- 1 

Sp*ei*l dabuf coda 


1 ' 

liij !Ji- 111! , M 




BHHHHHHI 



1 Rtadinv dooumtnttHon 

— 
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HHBHHHHI 

Proof taehniqu* 


mmmmmm 



Odwr lEaplain In Ei 

■ 





Whti wM ilw tinw uHd to iiolPd P«o cpumI 

how or Kmc ^.^oiw hour to on* day, __mor* than on* day, __n*y*r foutal 
If navar found, nraa a tvorkaround m*«i> v«« No (Eaplain In E) 

W*i mil arror ralaad to a piovioi'i cliwf 

V«« irh«ny H»nn»t 0/P«f t No Can't tall 

Whan did dt* arror antar dt* tyytami 

_r*ouirim*nt« __fu*ict«»»l ipaei ___d*il9n oodlnf and tan _oth*r ean't tall 

SECTION E - ADDITIONAL INFORMATION 

F<*af* gwa any Infonnation that may b* halpful in ealaporltinp Ih* arror or ehanpa, aril undantartdirtp id eauM and Ita 
ramiftcatlona, 


I 
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OF POOR 




QV. 


* 

u> 


Currtnt 0«t« 


Attitudt S/itm M«<ntan«nct Raport 


Projaet *Uma N aad for Chanpa dataralnad an C<o.. Day, rr.) 

Oaicrlba Changa 


Mhat eomponants/iubreutlnat/inodulas ara etian^ad 


CriAWGE tKow-ERi»OR) (fm out this ^act^on if chanoa is MOT an arror corraet^on) 
This changa It baing mada bacausa nf a changa In: (ChacEall that app’y) 

raqul ramants h ardwara anvlronmant 

naw Information/data t oftwara anvlronmant 

_____ ipael fi cation o otimliatlon 

_____ daiign 
_____ othar (ipaeify): 


ERROR oi<(LV out this taction if cfianga an arror corrtct^on) 

Tba folloMing activltlas wara utad in arrar datactlon or Isolation: (Chack all that 
apply) (Put 0 for datactlon. t for Isolation) 

traca/dunp 

cross rafaranca/attltuda list 
system error messages 
project specific error messages 


Which of the following best describes tha error: 

_____ raqui rements error s ped f1 cation error 

_____ design error c lerical error 

______ error In translating design or specification to coda 

_____ other: Describe 

Was this error related to a previous maintenance change v es n o can't tell 

Mease give any in/ormaflon that may be 'alp/ul In categorlimg and understatid(ng the 
cnange on the reverse side of this form. 

Parson filling out this form 

Approved D ate . 

Change started on date (month, day, year) 

Time spent on this change: 

less than 1 day 1 day to a v<eek m ore than a week 


_____ normal use 

______ test runs 

_____ coda reading 

______ raadino documentation 

_____ other (Specify): ___ 
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B.2 


SEL GLOSSARY OF TERMS USED WITH DATA COLLECTION FORMS 


This section defines the terms used in the software engi- 
neering data collection forms reproduced in Section B.l. A 
more extensive glossary (based substantially on this one) is 
found in Reference 19. 


assignment 

statements 


atti '.ude/orbit 


All statements that change the value of a 
variable as their main purpose (e.g.» as- 
signment or READ statements, but the as- 
signment of the DO loop variable in a DO 
statement should not be included) . 

Any component tnat is directly related to 
either the attitude determination (or con- 
trol) t£sk or to the orbit determination 
(or control) task falls into this cate- 
gory. This should include full systems in 
general (such as GTDS or ISEE-B Attitude) 
as well as specific modules such as Deter- 
ministic Attitude or DCCONES. 


attribute list 


automated 

tools 


baseline 

diagram 


A compiler -generated list of the identi- 
fiers used by a program that describes the 
characteristics of those identifiers and 
shows the source statements wliere they are 
first defined (or first used) and, for 
variables, their (relative) storage loca- 
tions. 

Any programs whose purpose is to aid in 
software development (e.g., compiler, text 
editor, or dump or trace facility). This 
includes compilers but not standard opera- 
ting system software (e.g., linkage edi- 
tor) . 

A structured chart listing all components 
in a system in which a connection from a 
higher component to a lower one indicates 
that the higher component calls the lower 
one. 


batch use of a computer in which the entire j^b 

is read into the machine before the proc- 
essing begins and in which there is no 
provision for interaction with the sub- 
mitter during execution of the job. (In- 
teractive usage is always via a terminal; 
batch usage may be via a terminal or a 
card deck.) 
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bottom-ap 


business/ 

financial 


change 


clerical 

code reading 

comirand/ 

control 

complexity 


component 


The design (or implementation) of the sys- 
tem starting with the lowest level rou- 
tines and proceeding to the higher level 
routines that use the lower levels. 

The second of the four major categories ap- 
plies to components related to some ac- 
counting task, financial data formatting, 
business data retrieval or reporting, or 
possibly personnel d^ita ranagement. Very 
few of the components being studied will 
fall into this class. 

A modxf icaticn to design, code, or docu- 
mentation. A change might be made to 
correct an errot, to improve system per- 
formance, to add capability, to improve 
appearance, or to implement a requirements 
change, for example. 

The process of copying an i;.em from one 
format to another or from one medium to 
another, which involves no interpretation 
or semantic translation. 

Visual inspection of the source code by 
persons other than the creator of the code. 

This class of components includes those 
used either to generate vehicle commands 
or to transmit these commands from the 
control center. 

Measures the difficulty of implementing a 
component, independent of the imple- 
menter's experience. !2asy (or simple) 
means that any good programmer can write 
down the correct code with little thought. 
Hard (or complex) means that much thought 
is involved in che design. (Compare this 
with ''precise;" e.g., easy and imprecise 
may mean a vague specification, but once 
the approach is decided upon, the code is 
easy to write. ) 

A piece of the system identified by name 
or common function (e.g., separately com- 
pilable function, an entry in a tree chart 
or baseline diagram for the system at any 
point in time, or a shared section of data 
such as a COMMON block) . 
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computer time 


confidence 

level 


constrainna 


constraints, 

space 


constraints, 

time 


control 

statements 


cor rect ion 
cosmetic 


create 

creation date 


cross- 

reference 


For batch usage, this is the billable time 
for all runs. For interactive usage, it 
is the number of hours spent at a terminal. 

Percentage probability that a given number 
is correct! 100 percent means that the 
number is absolute certainty; 0 percent 
means that the number must be incorrect. 

Restrivticns on resource availability (ex- 
ecution time, memory allocation) imposed 
by specifications. 

All restrictions caused by space problems. 
On the Component Summary Report form, list 
each restriction separately, e.g., maximum 
number of words that component may occupy 
at one time or maximum disk space avail- 
able during execution time or for program 
storage. 

All restrictions caused by various machine 
and calendar time problems. On the 
Component Summary Report form, list each 
restriction separately, e.g., maximum ex- 
ecution time for component to process and 
respond to some input condition or time to 
complete a component or milestone. 

All statements that potentially alter the 
sequence of executed instructions (e.g., 
GOTO, IF, RETURN, or DO). 

A change made to correct an error. 

Changes in the source program that have 
little effect on the performance of pro- 
gram, e.g., correct comments, move code 
around as long as it does not alter the 
algorithm implemented, or change the name 
of a local variable. 

The creation and recording of the idea. 

Date that the component was first named 
(e.g., date it first appeared on a tree 
chart) . 

A list of the identifiers used by a program 
showing (by means of indices or statement 
numbers) which statements of the program 
define and reference those identifiers. 
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data base 
applications 


design 


design phase 

design reading 

development 

phase 

documentation 


dump 


end date 

English (or 

informal) 

specifications 


This category is to include components that 
retrieve, write to, or format informat/.'^n 
for a well'defined formatted bank of in- 
formation available to the system. The 
user must decide whether the data set is 
to be considered a data base or not. An 
example of an acceptable data base would 
be the ADL file, SLP file, or Geodetics 
file, whereas a sequential telemetry file 
or tape would not be. 

A description of what the system must do, 
its components, the interfaces among those 
components, and the system's interface(s) 
to the external environment. 

The creation and recording of the design, 
including discussion about strategy with 
peers. This phase does not include the 
development of any code at the programming 
language level, it does include the crea- 
tion of specifications for subcomponents 
of the current component. 

Visual inspection of the design by persons 
other than the creator of the design. 

The development and recording of code and 
inline comments based on the design. This 
phase includes the modification of code 
caused by design changes or errors found 
in testing, it does not include any time 
spent in entering the code into the com- 
puter . 

Written material, other than source code 
statements, that describes a system or any 
of its components. 

A record of the state of the memory space 
used by a program at some point in its 
execution. A dump may include all or part 
of the program's memory space (including 
registers) . 

Date that a project is scheduled to be 
completed. 

Specifications given as readable English 
text, as opposed to some formal notation. 
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«r ror 


«xt«cnal 

environment 


formal spec- 
if icationa 


function 


functional 

specifications 


hardest first 


HIPO (Hier- 
aichical Input 
Process Output) 

implementation 


integration 

test 

integration 
test, full 


A discrepancy between a specification and 
its implementation. The specification 
might be requirements, design specifica- 
tions, or coding specifications. 

The combination of hardware and software 
used to maintain and execute the software, 
includi’.g the computer on which the soft- 
ware executes, the operating system for 
that computer, support libraries, text 
editors, and compilers. 

Some specification technique based upon a 
strict set of rules for describing the 
specification and usually involving the 
use of an unambiguously defined notation 
(e.g., mathematical functions or formal 
PDL) . 

A mathematical notation used to specify 
the set of input, :he set of output, and 
the relationship between input and output. 

A specification of a component as a set of 
functions defining the output for any in- 
put. The specification emphasizes what 
the program is to do rather than how to do 
it. However, an algorithmic specification 
can be considered functional if it is not 
used to dictate the actual algorithm to be 
used. (See procedural specifications.) 

The design (or implementation) of the most 
difficult aspects of the system first. 

A graphical technique that defines each 
component by its transformation on its 
input data sets to its output data sets. 

The implementation of a program is either 
a machine-executable form of the program, 
or a form of the program that can be auto- 
matically translated (e.g., by compiler or 
assembler) into machine-executable form. 

A test of several modules to check that 
the interfaces are defined correctly. 

Test of tho entire system (i.e., top- 
level component) . 
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integration 
teat* partial 

intended 
use of 


inter face 


interactive 


iterative 

enhancement 


level 


level, lowest 


librar ian 


machine words 


manpower 


Test of any set of modules but not the 
entire system. 

The result of invok;..g a program or segment 
of a program, including the actions per- 
formed by that program when invoked, in- 
vocation may be by subroutine or function 
call or by a branch to a segment of code. 

The set of data passed between two or more 
programs or segments of programs and the 
assumptions made by r ,ch program about how 
the others operate. 

Use of a computer via a terminal in which 
each line of input is immediately proc- 
essed by the computer. 

The design (or implementation) of succes- 
sive versions, eaoth producing a usable 
subset of the final product until the en- 
tire system is fully developed. 

A rait corresponding to some partitioning 
O' the final product (e.g., a single line 
o£ code, 10 lines of code, 25 lines of 
code, subroutine, or module) . If the sys- 
tem is hierarchically structured, each 
component is at a higher level than its 
subcomponents, and the .< ystem may be de- 
scribed as the highest level component 
(the component at level 1) , the component 
at level 2, or the lowest level component. 

Smallest unit identified by the activity 
(e.g., code reading to the single state- 
ment, top-down design to the module level, 
or top-down design to level 3) . 

A clerk whose responsibilities include 
processing source statements but not writ- 
ing them, (e.g., maintaining libraries, 
updating code, or producing tape backups). 

Number of words in a main memory that a 
component occupies at one time. 

The sum, over the number of people, of the 
number of hours per person charged to the 
contract. 
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mathematical/ 

numerical 


maximum apace 

mission date 
module test 
none used 

on-board 

proceitsing 


optimization 


PDL 


This category is meant to be a more speci- 
fic category than the scientific class. 

It contains those components that reflect 
a specific algebraic expression or mathe- 
matical algorithm. Such components as a 
dot product routine or a numerical inte- 
grator are in this category. 

Total ni mber of machine words that the 
system may occupy at one time. 

Date thkit system must be operational. 

Test of a single module. 

NO explicit technique was specified to be 
used. 

All components that are built for the 
purpose of satisfying some on-board proc- 
essing need belong to this class. Al- 
though the component may be built and 
tested on a computer that is not the real 
flight computer, it should be classified 
as onboard if the final destination is the 
OBC (onboard computer) . 

Changes in the source code to improve pro- 
gram performance, e.g., run faster or use 
less space. Optimization changes are not 
error corrections; however, if a change is 
made to use less space to conform to the 
specified space constraint, then the term 
"error" applies. 

A program design language (often called 
pseudocode) . used in the design and cod- 
ing phases of a project, PDL is a language 
that contains a fixed set of control state 
ments and a formal or informal way of de- 
fining and operating on data structures. 
PDL code may or may not be machine- 
readable, and Cor. this study it is not con 
sidered as documintation, but as an 
integral part of the finished source pro- 
gram. 
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procedural 
specif icationa 


proof 

technique 


range in mod- 
ule size 

read 


real-tl me 


req ?irements 


review 


scientific 


A specification of a component in some al- 

? orithmic manner (e.g., using PDL or a 
lowchart) . The specification says how 
the program is to work. (See functional 
specifications. ) 

A method for formally demonstrating that a 
piece of software performs according to 
its specifications. Proof techniques usu- 
ally use some form of mathematical nota- 
tion to describe the result of executing a 
program. 

The number of source statements in a 
modular including comments. 

The reading by peers of the recordings of 
the current phase to look for erroror in- 
vent tests, and so on. 

This class includes components that are a 
direct function of events occurring at, or 
near, the current time. Typical compo- 
nents would be the Attitude Control 
Monitors. Since parts of most of the te- 
lemetry processors are required to process 
data as it is received, they too .pay be 
considered real-time components. 

A system specification written by the user 
to define a system to a developer. The 
developer uses these specifications in 
designing, implementing, and testing the 
system. 

A formal meeting of several individuals 
for the purpose of explaining design (man- 
agement review) . Also includes the time 
spent in preparing for the review. All 
those attending a review should lint the 
components discussed in their own Compo- 
nent Summary Report for that week. 

A component may be in this category if it 
is related to some mathematical algorithm, 
engineering problem, law of physics, or 
celestial mechanics problem. Most of the 
full systems develOg^ed will fall into this 
category, whereas the various pieces of 
modules may fall into some of the ocher 
classes . 
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segment 


shared items 


simulating 

constructs 


source 

instructions 

source 

statements 


specii!ication 


specification, 

imprecise 


specification, 

precise 


A contiguous piece of code that is unnamed 
and, hence, cannot be referred to as a 
single entity in a program statement. A 
segment could be one or several lines of a 
subroutine, part of a data area, or an 
arbitrary contiguous section of memory. 

Data and programs, accessible by several 
components, such as COMMON blocks, ex- 
ternal files, and library subroutines. 

Statements that are used to simulate struc- 
turad control structures when the language 
to be used does not contain structured 
control structures. 

Sec source statements. 


All statements readable by and read by the 
compiler. This includes executable state- 
ments (e.g., assignment, IF, iind GO TO); 
nonexecutable statements (e.g., DIMENSION, 
REAL, and END); and comments. 

A description of the input, output, and es- 
sential function(s) to be performed by a 
component of the system. The specifica- 
tion is produced by the organization that 
is to develop the system; that is, at the 
top level, it can be thought of as the 
contractor's interpretation of the re- 
quirements . 

The input, output, and function of the com- 
ponent are loosely defined. Much of what 
is required is assumed rather than speci- 
fied. The specification relies heavily on 
programmer experience and verbal communi- 
cation to get an unambiguous interpreta- 
tion and a full understanding of what is 
needed. 

The input, output, and function of the com- 
ponent are well defined. There are under- 
lying assumptions not specified, but it is 
assumed that any programmer working on the 
project, with experience on a similar 
project, will understand these assump- 
tions. It is possible to arrive at an am- 
biguous interpretation or misunderstanding 


B-36 


specification , 
precise 
(Cont ’d) 


specification, 
very precise 


specification- 

driven 


standards 


start date 

string proces- 
sing 


structure- 

driven 


structure 
of data 


structured 

code 


of the specifications if the reader does 
not have enough experience with the prob- 
lem or does not obtain further verbal com- 
munication . 

A completely defined description of the 
input, output, and function of a compo- 
nent. The impiementer of a very precise 
specification need make few, if any, as- 
sumptions. It is almost, impossible to 
arrive at an ambiguous interpretation or 
misunderstanding of the specifications. 

using the specifications of the program to 
determine test data (e.g., test data is 
generated by examining the input/output 
requirements and specifications) . 

Any specifications that refer to the 
method of development of the source pro- 
gram itself, and not to the problem to be 
implemented (e.g., using structured code, 
at most 100-line subroutines, or all names 
prefixed with subsystem name) . 

Date on which initial work on a project 
began . 

This includes components that perform op 
erations on lists of characters. Norm- 
ally, this class is assumed to include 
functions of compilers, hash code string 
hook-up, and array comparisons. 

Using the structure of the program to de- 
termine test data (e.g., generating data 
to ensure that each branch of a program is 
executed at least once) . 

The organization of a composite data item 
consisting of several variables or other 
array items. Examples of such composite 
data items are arrays (both singly- and 
multiply-dimensioned) , strings, complex 
variables and constants, records on a disk 
file (each record containing several 
words) , and multiple-word entries in a 
table . 

The language supports structured control 
structures (e.g., a FORTRAN preprocessor). 
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systems 


system size 


table handler 


telemetry/ 

tracking 


testing phase 


top-down 


trace 


type of soft- 
ware 


By system-related software, one includes 
any package designed to affect, modify, 
extend, or change the normal available 
processing procedure of the operating sys- 
tem. This could include such components 
as error tracing or extended I/O such as 
DAIO. 

Total number of machine words needed for 
all instructions generated on the project 
plus apace for data, library routines, and 
other codo. This is the total size of the 
system without using any overlay structure. 

includes components that are specifically 
designed to generate or interpret informa- 
tion in a table format such as the Gener- 
alized Telemetry Processor. 

Includes all components that are spec- 
ifically required to Interface (either 
read, write, or format) with telemetry or 
tracking data. 

The design of tests, testing strategies, 
and the running of such tests. This phase 
does not include the writing of any code 
(even for debugging purposes) , which 
should be recorded under coding. 

The design (or implementation) of the sys- 
tem, starting with a single component, one 
level at a time, by expanding each compo- 
nent reference as an algorithm possibly 
calling other new components. 

A record of program execution showing the 
sequence of subroutine and function calls 
and, sometimes, the value of selected var- 
iables. code used in producing a trace is 
automatically inserted into a program, 
usually by the compiler, sometimes by 
other support software. 

The four major classifications of most of 
the applicable software being developed 
are: scientific, business/financial, 

systems, and utility. These classifica- 
tions may be refined into the categories 
of: string processing, data base 

applications, real time, and table 
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type cf soft- 
ware 
(Cont'd) 

utility 


value of data 


walkthrough 


workaround 


handler. A further refinement includes 
the categories of: attitude/orbit, 

telemetry/tracking, command/control ^ math- 
emati::al, and numerical on-boaru. 


Any component that is generated to satisfy 
some general support function required by 
other applications software may be con- 
sidered a utility. One thinks of this 
class of components as containing software 
that does not fit into any of the other 
three categories. Although components can 
fall into two of the primary categories 
(e.g., scientific and utility), it will be 
easier to use only the more descriptive of 
the categories (e.g., vector cross 
product--scientif ic; data unpacking-- 
utility) . 

The number and kind of number (e.g., in- 
teger, floating-point, or ASCII-encoded 
character) stored in a local variable or 
data area, parameter, common variable, or 
system-wide data item. 

Formal meeting sessions for the review of 
source code and design by the various mem- 
bers of the project for technical rather 
than management purposes. The purpose is 
for error detection and not correction. 

The method used to counteract the effects 
of an error in a program when the cause of 
the error and, consequently, the location 
of the statements containing the error is 
not known or is inaccessible (e.g., a com- 
piler error ) . 
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of the specifications if the re-der does 
not have enough experience with the prob- 
lem or does not obtain further verbal com- 
munication . 

A completely defined description of the 
input, output, and function of a compo- 
nent. The irplementer of a very precise 
specification need make few, if any, as- 
sumptions. It is almost impossible to 
arrive at an ambiguous interpretation or 
misunderstanding of the specifications. 

using the specifications of the program to 
determine test data (e.g., test data is 
generated by examining the input/output 
requirements and specifications) . 

Any specifications that refer to the 
method of development of the source pro- 
gram itself, and not to the problem to be 
implemented {e.g., using structured code, 
at most 100-line subroutines, or all names 
prefixed with subsystem name) . 

Date on which initial work on a project 
began. 

This includes co.uponents that perform op 
erations on lists of characters. Norm- 
ally, this class is assumed to include 
functions of compilers, ha&h code string 
hook-up, and array comparisons. 

using the structure of the program to de- 
termine test data (e.g., generating data 
to ensure that each branch of a program is 
executed at least once) . 

The organization of a composite data item 
consisting of several variables or other 
array items. Examples of such composite 
data items are arrays (both singly- and 
multiply-dimensioned), strings, complex 
variables and constants, records on a disk 
file (each record containing several 
words), and multiple-word entries in a 
table . 

The lanauage supports structured control 
structu.es (e.g., a FORTRAN preprocessor). 
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